Aws Data Engineer

4.0 - 8.0 years

8.5 - 13.0 Lacs P.A.

Hyderabad, Pune, Bengaluru

Posted:3 days ago| Platform: Naukri logo

Apply Now

Skills Required

HiveQLMWAA DAGsAWS DevelopersSparkSQLPySparkSQL

Work Mode

Hybrid

Job Type

Full Time

Job Description

Role Technology Lead No of years experience 5+ Detailed job description - Skill Set: Role Summary: As part of the offshore development team, the AWS Developers will be responsible for implementing ingestion and transformation pipelines using PySpark, orchestrating jobs via MWAA, and converting legacy Cloudera jobs to AWS-native services. Key Responsibilities: Write ingestion scripts (batch & stream) to migrate data from on-prem to S3. Translate existing HiveQL into SparkSQL/PySpark jobs. Configure MWAA DAGs to orchestrate job dependencies. Build Iceberg tables with appropriate partitioning and metadata handling. Validate job outputs and write unit tests. Required Skills: 35 years in data engineering, with strong exposure to AWS. Experience in EMR (Spark), S3, PySpark, SQL. Working knowledge of Cloudera/HDFS and legacy Hadoop pipelines. Prior experience with data lake/lakehouse implementations is a plus Mandatory Skills AWS Developer

Zentek Infosoft
Not specified
[ ]

RecommendedJobs for You

Pune, Bengaluru, Mumbai (All Areas)

Hyderabad, Pune, Bengaluru

Pune, Maharashtra, India

Hyderabad, Chennai, Bengaluru