Azure Data Engineer

Colan Infotech

3 - 5 years

10 - 15 Lacs

chennai tamil nadu india

Posted:10 hours ago| Platform: Foundit logo

Apply

Skills Required

scala spark data pipelines

Work Mode

On-site

Job Type

Full Time

Job Description

Responsibilities:

Design, develop, and maintain
scalable and efficient data pipelines
using
PySpark
,
Scala Spark
,
Databricks
,
Python
, and
SQL
.
Write
optimized, reusable, and high-quality code
for data processing and transformation.
Optimize
SQL queries
for high-performance data extraction, manipulation, and analysis.
Demonstrate strong expertise in
Databricks
, including
workflow management
,
job orchestration
, and
data exploration
.
Collaborate with
cross-functional teams
to gather and understand business and data requirements.
Implement
best practices
for
ETL
,
data pipeline optimization
, and
query performance tuning
.
Develop and maintain
comprehensive documentation
for all data pipelines, workflows, and related processes.
Troubleshoot, debug, and resolve data pipeline issues promptly to ensure
data accuracy
and
minimal downtime
.
Continuously explore opportunities for
automation
,
performance improvements
, and
scalability
enhancements in data workflows.

Requirements:

Strong programming skills in
Python
,
PySpark
, and
SQL
.
Experience with
Databricks
and
Spark
-based data processing frameworks.
Good understanding of
ETL design principles
,
data modeling
, and
data architecture
.
Hands-on experience with
workflow orchestration tools
and
version control systems
(e.g., Git).
Familiarity with
cloud-based data platforms
(AWS, Azure, or GCP) is an added advantage.

More Jobs at Colan Infotech

Senior Sharepoint Developer (SPFx, React)

Chennai

6 - 10 yrs

INR 15 - 25 Lacs

Postgresql Developer

Chennai

5 - 10 yrs

INR 0 - 0 Lacs

Urgent Vacancy For Business Development Executive

Chennai

1.0 - 4.0 yrs

INR 1 - 5 Lacs

GCP Data Engineer

Chennai

4.0 - 6.0 yrs

INR 10 - 15 Lacs

React Native Developer

Chennai

3.0 - 6.0 yrs

INR 6 - 12 Lacs

Mock Interview

Practice Video Interview with JobPe AI

Start Job-Specific Interview

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.