Senior Data Engineer

We are seeking a highly skilled and experienced Cloud Data Engineer to join our dynamic team. You will be responsible for designing, building, and maintaining scalable data pipelines and infrastructure on GCP/AWS/Azure, ensuring data is accessible, reliable, and available for business use.

Key Responsibilities:

• Data Pipeline Development:

  Design, develop, and maintain data pipelines using GCP/AWS/Azure services such as Dataflow, Dataproc, BigQuery, and Cloud Storage.

• Data Integration:

   Work on integrating data from various sources (structured, semi-structured, and unstructured) into GCP/AWS/Azure environments.

• Data Modeling:

   Develop and maintain efficient data models in BigQuery to support analytics and reporting needs.

• Data Warehousing:

   Implement data warehousing solutions on GCP, optimizing performance and scalability.

• ETL/ELT Processes:

   Build and manage ETL/ELT processes using tools like Apache Airflow, Data Fusion, and Python.

• Data Quality & Governance:

  Implement data quality checks, data lineage, and data governance best practices to ensure high data integrity.

• Automation:

   Automate data pipelines and workflows to reduce manual effort and improve efficiency.

• Collaboration:

   Work closely with data scientists, analysts, and other stakeholders to understand data requirements and deliver data solutions that meet business needs.

• Optimization:

   Continuously monitor and optimize the performance of data pipelines and queries for cost and efficiency.

• Security:

   Ensure data security and compliance with industry standards and best practices.

Required Skills & Qualifications:

• Education:

   Bachelors degree in Computer Science, Information Technology, Engineering, or a related field.

• Experience:

   8+ years of experience in data engineering, with at least 2 years working with GCP/Azure/AWS

• Technical Skills:

• Strong programming skills in Python, SQL,Pyspark and familiarity with Java/Scala.

• Experience with orchestration tools like Apache Airflow.
• Knowledge of ETL/ELT processes and tools.
• Experience with data modeling and designing data warehouses in BigQuery.
• Familiarity with CI/CD pipelines and version control systems like Git.
• Understanding of data governance, security, and compliance.

• Soft Skills:

• Excellent problem-solving and analytical skills.
• Strong communication and collaboration abilities.
• Ability to work in a fast-paced environment and manage multiple priorities.

Preferred Qualifications:

• Certifications:

   GCP Professional Data Engineer or GCP Professional Cloud Architect certification.

• Domain Knowledge:

   Experience in the finance, e-commerce, healthcare domain is a plus.