AWS Glue + PySpark

Cognizant

8 - 13 years

27 - 42 Lacs

hyderabad

Posted:18 hours ago| Platform:

Apply

Skills Required

configuration amazon redshift pyspark sql etl pipelines java spark data transformation etl etl development performance tuning workflow data processing query data engineering aws lambda aws glue performance partitioning troubleshooting athena agile aws ssis error handling

Work Mode

Work from Office

Job Type

Full Time

Job Description

Skill: AWS Glue + PySpark

Experience

Location : AIA - Pune

Key Responsibilities:

Design, develop, and optimize AWS Glue ETL jobs using PySpark.
Integrate data from multiple sources (S3, RDS, Redshift, DynamoDB, etc.) into a centralized data lake or warehouse.
Implement data transformations, cleansing, and enrichment logic in PySpark.
Configure and manage AWS Glue Crawlers for schema inference and Data Catalog updates.
Optimize job performance by tuning Spark configurations and partitioning strategies.
Implement error handling, logging, and monitoring for Glue jobs.
Collaborate with data engineers and analysts to define data requirements and transformation rules.
Ensure data quality, security, and compliance with organizational and regulatory standards.
Automate ETL workflows using AWS Glue Workflows, Triggers, and Step Functions.
Troubleshoot and resolve issues in production ETL pipelines.

Required Skills & Qualifications:

3+ years of experience in data engineering or ETL development.
Strong proficiency in PySpark and distributed data processing.
Hands-on experience with AWS Glue, AWS S3, AWS Lambda, AWS Athena, and AWS Redshift.
Solid understanding of Spark performance tuning and partitioning strategies.
Experience with AWS Glue Data Catalog and schema management.
Proficiency in SQL for querying and transforming data.
Familiarity with IAM roles, policies, and AWS security best practices.
Strong problem-solving skills and ability to work in an agile environment.

More Jobs at Cognizant

Workday: Walk-in drive 10th May Pune with CTS(Full time)

Pune, Mumbai (All Areas)

3 - 8 yrs

INR 7 - 17 Lacs

Java Full Stack Developer

Chennai

5 - 10 yrs

INR 10 - 20 Lacs

Walkin For Freshers @hyderabad

Hyderabad

Experience: Not specified

INR 0 - 2 Lacs

React JS Developer

Hyderabad

2 - 7 yrs

INR 9 - 14 Lacs

Java Developer

Bengaluru

5 - 10 yrs

INR 15 - 30 Lacs

Mock Interview

Practice Video Interview with JobPe AI

Start PySpark Interview

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.