Azure Data Engineer

Role & responsibilities

5 to 8 years of experience in data engineering or a related field.

At least 2 years of hands-on experience with Databricks, PySpark, Azure Synapse, and cloud platforms (preferably Azure).

Technical Skills

Strong expertise in Python/PySpark programming applied to data engineering.

Solid experience with cloud platforms (Azure preferred, AWS/GCP) and their data services.

Advanced SQL skills and familiarity with relational and NoSQL databases (e.g., MS SQL, MySQL, PostgreSQL).

Strong understanding of CI/CD pipelines and automation practices in data engineering.

Experience with large-scale data processing on cloud platforms.

Soft Skills

Strong analytical and problem-solving skills.

Excellent communication skills and the ability to work collaboratively across teams.

High attention to detail with a proactive approach to improving systems and processes.

Education:

Bachelors or masters degree in computer science, Engineering, Information Systems, or a related field.

Key Responsibilities

Data Infrastructure and Pipeline Development:

Design, develop, and maintain complex ETL/ELT pipelines using Azure Synapse.

Design, build, and maintain data pipelines and APIs in the cloud environment, with a focus on Azure cloud Platform.

Optimize data pipelines for performance, scalability, and cost efficiency.

Implement data governance, quality, and security best practices throughout data workflows.

Cloud Platform Management

Design and manage cloud-based data infrastructure on Azure and other cloud platforms.

Utilize cloud-native tools and services to enhance data storage and processing capabilities.

Build and manage CI/CD pipelines for data engineering projects to ensure smooth deployments and automation.

Programming and Automation

Write and maintain high-quality, reusable code in Azure Synapse environments for data processing and automation.