Data Engineer – Azure Databricks

Job Description:

We are looking for an experienced Data Engineer with strong expertise in Azure Databricks to join our team. The ideal candidate will have hands-on experience in building scalable data pipelines and working with cloud-based big data technologies.

Key Responsibilities:

• Design, develop, and maintain scalable data pipelines using Azure Databricks
• Work with Azure Data Lake, Azure Synapse, and other Azure data services
• Perform ETL/ELT operations to ingest and transform data from multiple sources
• Optimize and troubleshoot data workflows for performance and reliability
• Collaborate with data scientists, analysts, and other stakeholders to deliver high-quality data solutions
• Ensure data quality, security, and compliance throughout all processes

Required Skills:

• Strong experience with Azure Databricks, Spark, and PySpark
• Proficiency in SQL, Python, and data pipeline development
• Experience with Azure Data Factory, Azure Data Lake, and other Azure services
• Good understanding of data warehousing and data modeling concepts
• Familiarity with CI/CD and version control (e.g., Git, Azure DevOps)
• Strong problem-solving and communication skills

Preferred Qualifications:

• Experience in real-time data processing (Kafka, Event Hubs)
• Knowledge of Delta Lake and Data Lakehouse architecture
• Microsoft Azure certification is a plus

Job Types: Full-time, Permanent

Pay: ₹500,000.00 - ₹2,400,000.00 per year

Benefits:

• Health insurance
• Paid sick time

Ability to commute/relocate:

• Coimbatore, Tamil Nadu: Reliably commute or planning to relocate before starting work (Required)

Experience:

• Azure: 3 years (Required)

Work Location: In person