Azure Data Engineer

Role: Azure data engineer

Work Mode: 5 days

Location: Bengaluru (WFO)

Experience: 5+ years

Primary skills:

Candidate should have experience in 80% in azure data engineering and 20% in azure devops ci/cd

Job Description:

Key Responsibilities:

Maintain and monitor Databricks-based data pipelines and Azure data services (e.g., Data Lake, Synapse, Key Vault).

Implement CI/CD pipelines using Azure DevOps for data engineering workflows.

Conduct regular system health checks and performance tuning of clusters and jobs.

Automate infrastructure provisioning using tools like Terraform or ARM templates.

Use monitoring tools (e.g., Azure Monitor, Log Analytics, Prometheus) to track system metrics and alert on anomalies.

Perform security audits using tools like Databricks Security Analysis Tool (SAT) to ensure compliance with best practices.

Manage secrets and credentials securely using Azure Key Vault and Databricks secret scopes.

Collaborate with security teams to enforce network security, access control, and data encryption policies.

Maintain operational runbooks and incident response

Qualifications:

Bachelor's degree in computer science, Engineering, or related field.

5+ years of experience in DevOps, cloud operations, or data platform support.

Proficiency in Azure services, Databricks, and scripting languages (Python, Bash).

Experience with CI/CD tools (Azure DevOps, GitHub Actions) and infrastructure-as-code.

Familiarity with security frameworks and auditing tools in cloud environments.

Strong troubleshooting skills and a proactive approach to system reliability.

Preferred Skills:

Experience with Databricks SAT for security posture assessments.

Knowledge of Unity Catalog, RBAC, and data governance in Azure Databricks.

Exposure to MLOps and operationalizing machine learning pipelines.

Understanding of cost optimization strategies in Azure.

Job Type: Full-time

Pay: ₹700,000.00 - ₹1,700,000.00 per year

Benefits:

• Health insurance
• Provident Fund

Work Location: In person