Job
Description
In this role, you will be responsible for performing AWS Glue ETL development using Python and PySpark for data transformation. Your proficiency in AWS services such as S3, Redshift, Athena, Lambda, and EMR will be crucial. You must have experience working with data formats like Parquet, Avro, JSON, CSV, and ORC. Advanced SQL skills are required for data querying and transformation. Additionally, you should possess a deep understanding of ETL/ELT design patterns, best practices, and data modeling concepts including dimensional modeling and star/snowflake schemas. Experience with version control using Git/Bitbucket is important for code management. Key Responsibilities: - Hands-on experience in AWS Glue ETL development - Strong programming skills in Python and PySpark - Proficiency in AWS services like S3, Redshift, Athena, Lambda, and EMR - Experience with data formats such as Parquet, Avro, JSON, CSV, and ORC - Advanced SQL skills for data querying and transformation - Understanding of ETL/ELT design patterns and best practices - Knowledge of dimensional modeling and star/snowflake schemas - Experience with Git/Bitbucket for code management Qualifications Required: - 3+ years of hands-on experience in AWS Glue - Strong programming skills in Python and PySpark - Proficiency in AWS services like S3, Redshift, Athena, Lambda, and EMR - Experience with data formats such as Parquet, Avro, JSON, CSV, and ORC - Advanced SQL skills for data querying and transformation - Understanding of ETL/ELT design patterns and best practices - Knowledge of dimensional modeling and star/snowflake schemas - Experience with Git/Bitbucket for code management Please note that the company prefers candidates with experience in the Vanguard platform or financial services domain, knowledge of AWS Step Functions for workflow orchestration, familiarity with Apache Spark optimization techniques, experience with data lake architectures and Delta Lake, AWS certifications (Solutions Architect, Data Analytics, Developer), experience with CI/CD pipelines for data workflows, knowledge of data governance and compliance frameworks, and exposure to Terraform or CloudFormation for infrastructure as code. In this role, you will be responsible for performing AWS Glue ETL development using Python and PySpark for data transformation. Your proficiency in AWS services such as S3, Redshift, Athena, Lambda, and EMR will be crucial. You must have experience working with data formats like Parquet, Avro, JSON, CSV, and ORC. Advanced SQL skills are required for data querying and transformation. Additionally, you should possess a deep understanding of ETL/ELT design patterns, best practices, and data modeling concepts including dimensional modeling and star/snowflake schemas. Experience with version control using Git/Bitbucket is important for code management. Key Responsibilities: - Hands-on experience in AWS Glue ETL development - Strong programming skills in Python and PySpark - Proficiency in AWS services like S3, Redshift, Athena, Lambda, and EMR - Experience with data formats such as Parquet, Avro, JSON, CSV, and ORC - Advanced SQL skills for data querying and transformation - Understanding of ETL/ELT design patterns and best practices - Knowledge of dimensional modeling and star/snowflake schemas - Experience with Git/Bitbucket for code management Qualifications Required: - 3+ years of hands-on experience in AWS Glue - Strong programming skills in Python and PySpark - Proficiency in AWS services like S3, Redshift, Athena, Lambda, and EMR - Experience with data formats such as Parquet, Avro, JSON, CSV, and ORC - Advanced SQL skills for data querying and transformation - Understanding of ETL/ELT design patterns and best practices - Knowledge of dimensional modeling and star/snowflake schemas - Experience with Git/Bitbucket for code management Please note that the company prefers candidates with experience in the Vanguard platform or financial services domain, knowledge of AWS Step Functions for workflow orchestration, familiarity with Apache Spark optimization techniques, experience with data lake architectures and Delta Lake, AWS certifications (Solutions Architect, Data Analytics, Developer), experience with CI/CD pipelines for data workflows, knowledge of data governance and compliance frameworks, and exposure to Terraform or CloudFormation for infrastructure as code.
