Design and develop robust, scalable data pipelines using
PySpark
on
Azure Databricks
.
Implement ETL/ELT processes to ingest, transform, and load data from various sources into data lakes or data warehouses.
Collaborate with data scientists, analysts, and business stakeholders to understand data requirements and deliver solutions.
Optimize data workflows for performance, reliability, and cost-efficiency.
Ensure data quality, integrity, and governance across all data platforms.
Monitor and troubleshoot data pipelines and workflows in production environments.
Work with Azure services such as
Data Lake Storage
,
Azure Synapse
,
Azure Data Factory
, and
Azure DevOps
.
Maintain documentation for data architecture, processes, and best practices.

Preferred candidate profile

Required Skills & Qualifications:

Bachelors or Masters degree in Computer Science, Engineering, or related field.
3+ years of experience in data engineering or big data development.
Strong proficiency in
PySpark
and
Apache Spark
.
Hands-on experience with
Azure Databricks
and other Azure data services.
Proficient in SQL and working with relational and non-relational databases.
Experience with CI/CD pipelines and version control (e.g., Git, Azure DevOps).
Familiarity with data modelling, data warehousing, and performance tuning.
Excellent problem-solving and communication skills.

Preferred Qualifications:

Experience with
Delta Lake
,
MLflow
, or
Unity Catalog
.
Knowledge of
data governance
and
security best practices
in cloud environments.
Certifications in Azure (e.g., Azure Data Engineer Associate) are a plus

More Jobs at Tuteck

React Developer

kolkata

3.0 - 8.0 yrs

INR 4 - 6 Lacs

Looking For Data Engineer - Immediate Joiner

kolkata, bengaluru

4.0 - 9.0 yrs

INR 15 - 30 Lacs

Mock Interview

Practice Video Interview with JobPe AI

Start PySpark Interview

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.