Big Data Lead / AWS

Job Summary

Seeking an experienced ETL Developer to design, build, and maintain data pipelines and integrations across cloud-native environments. The role focuses on developing robust ETL processes, ensuring data quality, performance, and scalability using AWS services and Python.

Priority Skills

• ETL knowledge
• AWS tech stack (AWS Lambda, AWS S3, AWS Terraform)
• AWS Glue
• Python

Secondary Skills (Good to Have)

• GraphQL
• PostgreSQL
• GitHub Actions

Location: Chennai

VimalaP3@hexaware.com

https://lnkd.in/daMht8-A

Key Responsibilities

• Design, develop, test, and deploy ETL pipelines to ingest, transform, and load data from multiple sources
• Implement cloud-native data solutions leveraging AWS services and infrastructure-as-code
• Maintain and optimize AWS Glue jobs, Lambda functions, and S3-based data lakes
• Write clean, maintainable Python code for data processing and orchestration
• Collaborate with data engineers, analysts, and stakeholders to translate requirements into technical solutions
• Monitor, troubleshoot, and tune ETL performance and data quality issues
• Create and maintain pipeline documentation, data lineage, and operational runbooks
• Implement CI/CD for data pipelines and automation around deployments and testing
• Ensure security, governance, and cost-efficiency of data processing workflows

Required Qualifications

• Bachelors degree in Computer Science, Engineering, Information Systems, or equivalent experience
• Proven experience designing and building ETL/data pipelines in production
• Strong Python development skills and experience with data libraries (e.g., pandas, pyarrow)
• Hands-on experience with AWS services listed in Priority Skills
• Familiarity with infrastructure-as-code (Terraform) and deployment automation
• Solid understanding of data modeling, data warehousing concepts, and data quality best practices

Desired Experience & Traits

• Experience with AWS Glue job authoring, debugging, and performance tuning
• Experience working with data lakes, partitioning strategies, and S3 lifecycle/cost management
• Knowledge of API integrations and GraphQL endpoints
• Comfortable working in agile teams and collaborating across engineering and data functions
• Strong problem-solving, communication, and documentation skills