(Remote) Azure Data Engineer

Key Responsibilities

• Design, develop, and maintain scalable data pipelines and ETL/ELT processes using Azure Databricks and other Azure services such as Azure Data Factory and Azure Data Lake Storage.
• Collaborate closely with user teams, data scientists, business analysts, and other stakeholders to understand data needs and translate them into effective technical solutions and use cases.
• Implement data integration, transformation, and validation pipelines to support analytics and reporting.
• Develop data models and data warehouses or lakes that support specific user use cases.
• Manage and configure Databricks workspaces, clusters, jobs, and data access controls (such as Unity Catalog).
• Automate data pipeline, deployment, scaling, and monitoring of data solutions using scripting and cloud-native tools.
• Document technical designs, processes, and solutions for knowledge sharing.

Requirements

• Bachelor’s degree in Computer Science, Engineering, or a related field.
• Solid experience as a Data Engineer, particularly with Azure Databricks.
• Proficiency in SQL, PySpark, Python, and ETL tools.
• Knowledge of data warehousing, big data technologies, and data modeling concepts.
• Familiarity with Azure cloud service. Experience with Databricks features such as Delta Lake, Spark SQL, cluster management, and performance tuning.
• Experience with data governance, compliance, and security best practices.

Examples of Specific Use Case Support

• Designing and deploying scalable analytics pipelines that handle streaming and batch data.
• Providing onboarding and technical support for new data-driven user cases.
• Using tools like Power BI to visualize insights derived from processed data.

Job Type: Full-time

Pay: ₹560,759.21 - ₹1,942,973.83 per year