Data Engineer (DW)

Job Title:

Location:

Job Type:

Experience Required:

Company:

How to Apply

Job Summary:

BitsAtom Technologies is seeking a skilled and experienced Data Engineer to join our growing team. The ideal candidate will bring 7+ years of experience in data engineering with strong expertise in Azure, SSIS, Databricks, Python, and data pipeline architecture. The role involves leading technical initiatives, building scalable and reliable data systems, and working closely with cross-functional teams to support business analytics and insights.

Key Responsibilities:

• Design, build, and maintain scalable ETL/ELT pipelines using Azure Data Factory, Databricks, PySpark, and SQL.
• Engineer data workflows integrating structured and unstructured data from Azure Data Lake, Synapse, SQL Server, and external APIs.
• Optimize data storage and compute performance in Databricks and Delta Lake environments.
• Implement data modeling and transformation logic aligned with analytics, reporting, and machine learning requirements.
• Lead end-to-end data solutioning on Azure with a focus on performance and availability.
• Collaborate with data scientists, business analysts, and stakeholders to gather requirements and deliver validated datasets.
• Develop CI/CD pipelines using Azure DevOps or GitHub Actions.
• Ensure compliance with data governance, security, and regulatory standards.
• Monitor, troubleshoot, and resolve issues in data pipelines.
• Mentor junior engineers and provide technical leadership.

Required Qualifications:

• Bachelor’s degree in Computer Science, Engineering, or a related field (or equivalent work experience).
• 7+ years of professional experience in data engineering.
• Deep expertise in Databricks (Delta Lake, orchestration, notebooks).
• Strong experience with Azure Cloud Services (ADF, ADLS Gen2, Key Vault).
• Proficiency in Python, PySpark, and SQL.
• Strong understanding of data lakehouses, ETL/ELT architecture, and orchestration.
• Excellent communication and collaboration skills.
• Ability to translate complex technical data concepts to business language.

Preferred Qualifications:

• Experience with data mesh architecture.
• Familiarity with event-driven systems (Azure Event Hubs, Kafka).
• Exposure to MLflow, Unity Catalog, or Databricks Workflows.
• Experience with Power BI or similar BI tools.
• Familiarity with Terraform or Infrastructure-as-Code tools.

Certifications (Preferred):

• Azure Data Engineer Associate (DP-203)
• Databricks Certified Data Engineer