Data Engineer

Agilisium

5 - 8 years

9 - 19 Lacs

Hyderabad

Posted:2 months ago| Platform:

Apply

Skills Required

Pyspark AWS Data Bricks Python SQL

Work Mode

Work from Office

Job Type

Full Time

Job Description

Data Engineer

Role & responsibilities

Data Pipeline Development: Design, build, and maintain scalable data pipelines for ingesting, processing, and transforming large datasets from diverse sources into usable formats.
Performance Optimization: Optimize data processing and storage systems for cost efficiency and high performance, including managing compute resources and cluster configurations.
Automation and Workflow Management: Automate data workflows using tools like Airflow, Databricks APIs, and other orchestration technologies to streamline data ingestion, processing, and reporting tasks.
Data Quality and Validation: Implement data quality checks, validation rules, and transformation logic to ensure the accuracy, consistency, and reliability of data.
Cloud Platform Management: Manage and optimize cloud infrastructure (AWS, Databricks) for data storage, processing, and compute resources, ensuring seamless data operations.

Preferred candidate profile

Strong proficiency in
Python
for scripting and data manipulation.
Hands-on experience with
PySpark
for distributed data processing.
Proficient in writing complex
SQL
queries for large-scale data extraction and transformation.
Solid understanding and experience with
AWS
cloud ecosystem (especially S3, Glue, EMR, Lambda).
Knowledge of data warehousing, data lakes, and ETL/ELT processes.
Familiarity with version control tools like Git and workflow orchestration tools (e.g., Airflow) is a plus.

Location - Hyderabad (Work From Office)

Notice - Immediate or 15days

More Jobs at Agilisium

Scrum Master- Life Sciences (Commercial)

Chennai

10 - 14 yrs

INR 7 - 16 Lacs

Senior Customer Success Manager

Bengaluru

2 - 7 yrs

INR 8 - 13 Lacs

AI/ML Engineer

Hyderabad, Chennai

4.0 - 9.0 yrs

INR 9 - 18 Lacs

Fullstack Architect

Chennai

9.0 - 14.0 yrs

INR 20 - 30 Lacs

Life sciences - Sr.Project Manager

Chennai

10.0 - 17.0 yrs

INR 8 - 18 Lacs

Mock Interview

Practice Video Interview with JobPe AI

Start PySpark Interview

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.