Data Engineer

Job Title

Job Nature

Experience

Position Overview

Data Engineer

Key Responsibilities

• Design and develop ETL/ELT pipelines for both batch and real-time data processing.
• Build and optimize data models and data lake/warehouse architectures to support analytics and machine learning.
• Ensure data quality, governance, and security across all workflows.
• Collaborate with data scientists, analysts, and architects to enable data-driven solutions.
• Integrate data from diverse sources such as databases, APIs, text files and streaming platforms.
• Monitor, troubleshoot and optimize data pipelines for performance and scalability.
• Contribute to best practices, documentation, and reusable components.

Requirements

• 5+ years of experience in data engineering or related fields.
• Solid knowledge of SQL and relational databases
• Strong programming skills, preferably in Python and experience with Spark big data processing framework.
• Hands-on experience with cloud data services (AWS, Azure, or GCP).
• Experience with data pipeline orchestration tools (e.g., Airflow, Azure Data Factory, Step Functions).
• Familiarity with source code version control and CI/CD pipelines for automatic deployment
• Strong problem-solving skills and ability to thrive in a fast-paced, cloud-native environment.

Nice to have

• Experience with MS Fabric (lakehouse, notebooks, pipelines, etc)
• Experience with Databricks (lakeflow, declarative pipelines, notebooks, etc)
• Experience migrating data from SAP HANA database
• Knowledge of data governance and cataloging (e.g., Purview, Glue, Unity Catalog).
• Familiarity with machine learning pipelines and collaboration with data science teams.
• Databricks Certified Data Engineer Associate/Professional
• Microsoft Certified: Azure Data Engineer Associate

Qualifications

• Bachelor’s degree in computer science, IT, or a related field.

Work Location

Time Zone

• Selected candidate is required to work as per India Time (IST) / European Time (CET/GMT) / Americas Time Zone (EST)