AWS Data

Primary Skills

SQLAWSJDDesign, develop, and maintain scalable ETL pipelines using AWS Glue, PySpark, and AWS Lambda.Migrate structured and semi-structured data from existing data lakes (e.g., S3, Hadoop) to Amazon Redshift.Optimize data ingestion and transformation processes for performance and cost-efficiency.Collaborate with data architects, analysts, and business stakeholders to understand data requirements and ensure data quality.Implement data validation, cleansing, and transformation logic.Monitor and troubleshoot data pipelines and workflows.Ensure data security and compliance with organizational and regulatory standards.Document data flows, architecture, and processes.ResponsibilityDesign, develop, and maintain scalable ETL pipelines using AWS Glue, PySpark, and AWS Lambda.Migrate structured and semi-structured data from existing data lakes (e.g., S3, Hadoop) to Amazon Redshift.Optimize data ingestion and transformation processes for performance and cost-efficiency.