Databricks Data Engineer

Position Overview
We are seeking an experienced Data Engineer with strong expertise in Databricks, Azure Data Services ecosystem (ADF, ADLS, Microsoft Fabric, Synapse, etc.), Python, PySpark, and databases. The ideal candidate will be responsible for designing, building, and optimizing scalable data pipelines and solutions to enable advanced analytics, business intelligence, and AI-driven initiatives.

Key Responsibilities

• Design, develop, and maintain data pipelines using Azure Data Factory (ADF) and Databricks.
• Work with Azure Data Lake Storage (ADLS), Microsoft Fabric, and Azure Synapse Analytics to build robust data platforms.
• Develop scalable ETL/ELT solutions leveraging Python, PySpark, and SQL.
• Optimize data workflows for performance, reliability, and cost efficiency.
• Collaborate with data scientists, analysts, and business teams to ensure data availability and quality.
• Implement best practices for data governance, security, and compliance in the Azure ecosystem.
• Monitor, troubleshoot, and enhance data processes and pipelines in production environments.
• Stay current with evolving Azure cloud technologies, big data frameworks, and modern data architectures.

Required Skills & Qualifications

• Strong experience with Azure Data Services:
• Azure Data Factory (ADF)
• Azure Data Lake Storage (ADLS)
• Microsoft Fabric (nice-to-have but preferred)
• Azure Synapse Analytics
• Expertise in Databricks for data engineering and analytics.
• Proficiency in Python and PySpark for large-scale data processing.
• Strong knowledge of SQL and relational/non-relational databases.
• Experience with data modeling, ETL/ELT pipelines, and data warehousing concepts.
• Understanding of CI/CD, version control (Git), and DevOps practices.
• Excellent problem-solving, communication, and collaboration skills.

Preferred Qualifications

• Experience with data governance frameworks and security best practices in Azure.
• Knowledge of machine learning integration within Databricks.
• Familiarity with Power BI or other visualization tools.
• Prior experience in agile environments.

Education & Experience

• Bachelor’s or Master’s degree in Computer Science, Information Systems, Data Engineering, or related field.
• 3+ years of professional experience in data engineering or big data roles.

Job Type: Contractual / Temporary
Contract length: 6 months

Pay: ₹600,000.00 - ₹800,000.00 per year

Work Location: In person