Vanguard AWS Glue Developer

Job Title: Vanguard AWS Glue Developer

Location:

Experience:

Employment

Work Mode:

About the Role:

We are seeking an experienced AWS Glue Developer with strong expertise in building scalable ETL pipelines and data integration solutions on AWS cloud platform. The ideal candidate will have hands-on experience with Vanguard applications and AWS Glue ecosystem, capable of designing, developing, and optimizing large-scale data workflows.

Key Responsibilities:

Design, develop, and maintain ETL pipelines using AWS Glue for data ingestion, transformation, and loading

Build and optimize Glue jobs using Python/PySpark for processing large-scale datasets

Develop data integration solutions connecting multiple data sources to AWS data lakes and warehouses Implement data quality checks, validation rules, and error handling mechanisms in ETL workflows Work with Glue Data Catalog for metadata management and schema evolution

Optimize Glue job performance through partitioning, bucketing, and resource tuning

Collaborate with data engineers, architects, and business teams to understand data requirements Implement data security best practices including encryption, IAM policies, and access controls Troubleshoot and resolve production issues in ETL pipelines

Create technical documentation for ETL processes and data lineage Participate in code reviews and ensure adherence to coding standards

Monitor and maintain data pipeline health using CloudWatch and other AWS monitoring tools

Must-Have Skills:

Technical Skills (Mandatory):

• AWS Glue:

  3+ years of hands-on experience in AWS Glue ETL development

• Python/PySpark:

  Strong programming skills in Python and PySpark for data transformation

• AWS Services:

  Proficiency in S3, Redshift, Athena, Lambda, and EMR

• Data Formats:

  Experience with Parquet, Avro, JSON, CSV, and ORC file formats

• SQL:

  Advanced SQL skills for data querying and transformation

• ETL

  Concepts:

  Deep understanding of ETL/ELT design patterns and best practices

• Data Modeling:

  Knowledge of dimensional modeling, star/snowflake schemas

• Version Control:

  Experience with Git/Bitbucket for code management

Preferred Skills:

• Experience with Vanguard platform or financial services domain Knowledge of AWS Step Functions for workflow orchestration Familiarity with Apache Spark optimization techniques
• Experience with data lake architectures and Delta Lake
• AWS certifications (Solutions Architect, Data Analytics, Developer) Experience with CI/CD pipelines for data workflows
• Knowledge of data governance and compliance frameworks
• Exposure to Terraform or CloudFormation for infrastructure as code

Soft Skills:

• Strong analytical and problem-solving abilities
• Excellent communication skills for stakeholder interaction Ability to work independently and in team environments

Proactive approach to identifying and resolving issues Strong documentation skills

Educational Qualification:

Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or related field Relevant AWS certifications are a plus