Data Engineer - Azure

Data Engineer (Azure Databricks) -

Experience

Responsibilities:

• Develop and maintain scalable data pipelines using Databricks (PySpark, Delta Lake).
  
• Integrate data from multiple APIs (Azure Monitor, Azure Cost Management, Log Analytics, Purview).
  
• Implement ETL/ELT processes for structured and unstructured data.
  
• Ensure data quality, reliability, and observability across pipelines.
  
• Optimize performance (cost, CPU, and memory usage).
  
• Collaborate with frontend teams to expose APIs for dashboards.
  
• Maintain documentation and version control (Git/DevOps).
  

Required Skills:

• Strong experience with Azure Data Services (Databricks, ADLS, Synapse optional).
  
• Proficiency in Delta Lake, PySpark, and SQL.
  
• Experience with API integration (REST APIs, JSON, authentication).
  
• Familiarity with Azure Monitor, Cost Management, and Log Analytics.
  
• CI/CD knowledge (Azure DevOps preferred).
  
• Strong debugging and performance optimization skills.

ramyasrikarthika@infinitiresearch.com