Data Engineer

Atyeti

7 - 12 years

25 - 40 Lacs

bengaluru

Posted:6 days ago| Platform:

Apply

Skills Required

spark airflow pyspark data pipeline oops data engineer aws python sql

Work Mode

Hybrid

Job Type

Full Time

Job Description

This position will be responsible for design, build and maintenance of data pipelines running on Airflow, Spark on the AWS Cloud platform.

Roles and Responsibility:

Build and maintain all facets of Data Pipelines for Data Engineering team.
Build the pipelines required for optimal extraction, transformation, and loading of data from a wide variety of data sources using Spark, Python and Airflow.
Work with internal and external stakeholders to assist with data-related technical issues and data quality issues.
Engage in proof of concepts, technical demos and interaction with customers and other technical teams
Participate in the agile ceremonies
Ability to solve complex data-driven scenarios and triage towards defects and production issues

Technical Skills

Must Have Skills:

Proficient with Python, PySpark and Airflow
Strong understanding of Object-Oriented Programming and Functional Programming paradigm
Must have experience working with Spark and its architecture
Knowledge of Software Engineering best practices
Advanced SQL knowledge (preferably Oracle)
Experience in processing large amounts of structured and unstructured data, including integrating data from multiple sources.

Good to Have Skills:

Knowledge of Data’ related AWS Services
Knowledge of GitHub and Jenkins
Automated testing

More Jobs at Atyeti

Lead/ Senior Angular Developer ( Exp: 6+ To10 Years)

Hyderabad, Pune, Bengaluru

5 - 10 yrs

INR 15 - 20 Lacs

Senior Gresham CTC Developer

Pune, Chennai

4 - 8 yrs

INR 10 - 20 Lacs

Senior TLM Developer

Pune, Chennai

4 - 9 yrs

INR 15 - 20 Lacs

Lead Python SDET (Exp: 5+ To 8 Years )

Pune

4 - 9 yrs

INR 15 - 20 Lacs

Senior Intellimatch Developer

Pune, Chennai

4 - 8 yrs

INR 15 - 20 Lacs

Mock Interview

Practice Video Interview with JobPe AI

Start PySpark Interview

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.