AWS Data Engineer

Job Description

Role & responsibilities Experience: 6-9 years as a Data Engineer with a strong focus on PySpark and large scale data processing. PySpark Expertise: Decent to proficient in writing optimized PySpark code+ including working with DataFrames+ Spark SQL+ and performing complex transformations. AWS Cloud Proficiency: Fair experience with core AWS services+ such as S3+ Glue+ EMR+ Lambda+ and Redshift+ with the ability to manage and optimize data workflows on AWS. Performance Optimization: Proven ability to optimize PySpark jobs for performance+ including experience with partitioning+ caching+ and handling skewed data. Problem Solving Skills: Strong analytical and problem solving skills+ with a focus on troubleshooting data issues and optimizing performance in distributed environments. Communication and Collaboration: Excellent communication skills to work effectively with cross functional teams and clearly document technical processes. Added advantage AWS Glue ETL: Hands on experience with AWS Glue ETL jobs+ including creating and managing workflows+ handling job bookmarks+ and implementing transformations. Database Good working knowledge of Data warehouse like Redshift. Hands on experience with AWS Glue ETL jobs+ including creating and managing workflows+ handling job bookmarks+ and implementing transformations. Database Good working knowledge of Data warehouse like Redshift.