Consultant Specialist

Key Skills:

Roles & Responsibilities:

• Evaluate and provide scalable technical solutions to address complex and interdependent data processes.
• Ensure data quality and accuracy by implementing data quality checks, data contracts, and governance processes.
• Collaborate with software development teams and business analysts to understand data requirements and deliver fit-for-purpose data solutions.
• Lead the team in delivering end-to-end data engineering solutions.
• Design, develop, and maintain complex applications to support data processing workflows.
• Develop and manage data pipelines and workflows using Apache Airflow on GCP.
• Integrate data from various sources into Google BigQuery and Google Cloud Storage (GCS).
• Write and optimize advanced SQL queries for ETL/ELT processes.
• Maintain data consistency and troubleshoot issues in data workflows.
• Create and maintain detailed technical documentation for pipelines and workflows.
• Mentor junior data engineers and provide technical leadership and support.
• Lead project planning, execution, and successful delivery of data engineering initiatives.
• Stay updated with emerging trends and technologies in data engineering and cloud computing.

Experience Requirement:

• 6-8 yeras of experience in leading the design, development, and deployment of complex data pipelines.
• Strong working knowledge of Apache Airflow on GCP for orchestration.
• Hands-on experience integrating data into Google BigQuery and GCS from various sources.
• Proficient in writing and optimizing complex SQL queries for large-scale data processing.
• Practical knowledge of containerization technologies like Docker and Kubernetes.
• Experience in implementing data governance and adhering to data security best practices.
• Familiarity with Agile methodology and working in cross-functional teams.
• Experience with CI/CD pipelines and DevOps practices for data engineering workflows.

Education: