Job
Description
We are seeking a highly skilled Senior Engineer with expertise in ETL processes, SQL, GCP (Google Cloud Platform), and Python. As a Senior Engineer, you will play a key role in the design, development, and optimization of data pipelines and workflows that drive business insights and analytics. You will collaborate closely with cross-functional teams to ensure data systems are scalable, robust, and efficient. Key Responsibilities : Design & Develop ETL Pipelines : Build, maintain, and optimize scalable ETL workflows to ingest, transform, and load large datasets using best practices. Database Management : Write efficient, optimized SQL queries to extract, manipulate, and aggregate data from relational databases. Design and implement database schemas for optimal performance. Cloud Infrastructure : Utilize Google Cloud Platform (GCP) services, such as BigQuery, Dataflow, Pub/Sub, and Cloud Storage, to develop and manage cloud-based data solutions. Automation & Scripting : Use Python to automate processes, build custom data transformation logic, and integrate with various data systems and services. Performance Tuning : Ensure the performance of data pipelines and queries are optimized for speed and cost. Troubleshoot issues, implement best practices, and improve system performance. Collaboration & Mentorship : Collaborate with data analysts, data scientists, and other stakeholders to understand data requirements. Provide mentorship and guidance to junior engineers. Data Quality & Governance : Ensure high-quality data through validation, error handling, logging, and monitoring of ETL processes. Implement data governance best practices to maintain consistency and integrity. Documentation & Reporting : Document data pipeline designs, coding standards, and best practices. Create reports for stakeholders to provide insights into data processing activities. Required Skills and Qualifications : ETL Expertise : Strong experience in building, deploying, and optimizing ETL processes with tools like Apache Airflow, Dataflow, or custom Python scripts. SQL Proficiency : Advanced SQL skills with experience in writing complex queries, optimizing performance, and working with large datasets. Cloud Platform (GCP) : Deep understanding and hands-on experience with Google Cloud Platform, specifically BigQuery, Cloud Storage, Pub/Sub, Dataflow, and other data-related services. Python : Proficient in Python, especially for data manipulation, ETL automation, and integration with cloud-based solutions. Data Engineering Best Practices : Familiarity with modern data engineering frameworks, version control, CI/CD pipelines, and agile methodologies. Problem Solving : Strong analytical and troubleshooting skills with a focus on identifying solutions to complex data engineering challenges. Communication : Excellent communication skills, able to work effectively in a team and engage with non-technical stakeholders. Preferred Qualifications : Experience with other cloud platforms such as AWS or Azure. Knowledge of data lake and data warehouse architectures. Familiarity with containerization (Docker, Kubernetes) and orchestration tools. Understanding of data privacy, security, and compliance best practices. Education & Experience : Bachelors degree in Computer Science, Engineering, or a related field (Master’s preferred). 5+ years of experience in data engineering or a related technical field. Proven experience in designing and implementing data solutions in a cloud environment. Role & responsibilities
