Data Engineer

Agilisium

5 - 10 years

8 - 18 Lacs

hyderabad chennai

Posted:3 hours ago| Platform:

Apply

Skills Required

spark aws python sql pysp airflow pyspark databricks

Work Mode

Work from Office

Job Type

Full Time

Job Description

Key Responsibilities

Design, develop, and maintain scalable and efficient
data pipelines
for batch and real-time processing.
Work extensively with
Databricks and PySpark
to process and transform large datasets.
Write complex and optimized
SQL queries
for data extraction, transformation, and analysis.
Develop robust
ETL/ELT processes
to integrate data from various structured and unstructured sources.
Collaborate with data scientists, analysts, and business stakeholders to ensure reliable and accessible data.
Implement best practices for
data quality, governance, and security
on the AWS cloud platform.
Monitor and troubleshoot data workflows to ensure high availability and performance.
Contribute to the
automation of data processes
and CI/CD integration.

Required Skills & Qualifications

Proven experience as a
Data Engineer
(typically 38 years, depending on level).
Strong hands-on experience with
Python
for data engineering tasks.
Proficiency in
SQL
(writing complex queries, performance tuning).
Strong expertise in
PySpark
and
Databricks
.
Hands-on experience working in
AWS cloud environment
(S3, Glue, EMR, Redshift, Lambda, etc.).
Experience with data pipeline orchestration tools (e.g., Airflow, AWS Step Functions).
Solid understanding of
data modeling, warehousing, and ETL/ELT concepts
.
Knowledge of version control systems (Git) and CI/CD practices.
Excellent problem-solving and communication skills.

More Jobs at Agilisium

Scrum Master- Life Sciences (Commercial)

Chennai

10 - 14 yrs

INR 7 - 16 Lacs

Senior Customer Success Manager

Bengaluru

2 - 7 yrs

INR 8 - 13 Lacs

AI/ML Engineer

Hyderabad, Chennai

4.0 - 9.0 yrs

INR 9 - 18 Lacs

Fullstack Architect

Chennai

9.0 - 14.0 yrs

INR 20 - 30 Lacs

Life sciences - Sr.Project Manager

Chennai

10.0 - 17.0 yrs

INR 8 - 18 Lacs

Mock Interview

Practice Video Interview with JobPe AI

Start Python Interview

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.