Sr. Data Engineer

As a skilled professional with over 5 years of experience in programming with Python, you will be responsible for designing, developing, and implementing performant ETL pipelines using Python API (pySpark) of Apache Spark on AWS EMR. Your role will involve hands-on experience in developing ETL data pipelines, configuring EMR clusters on AWS, and working with AWS S3 object storage from Spark. You should have a strong proficiency in Python, familiarity with functional programming concepts, and data modeling. Your key responsibilities will include understanding Spark's Data frame and API, troubleshooting Spark jobs, and monitoring spark jobs using Spark UI. The ideal candidate should have at least 3 years of hands-on experience in developing ETL data pipelines using pySpark on AWS EMR and a good understanding of Spark's Data frame and API. This is a full-time, permanent position suitable for someone with a solid background in ETL, Python programming, Apache Spark, pySpark, AWS S3, and Data Modeling. The work location is in person, and the application deadline is 22/02/2025. Benefits include the flexibility to work from home, and the schedule consists of day shift, fixed shift, and morning shift. If you have a strong background in ETL, Python, Apache Spark, pySpark, AWS S3, and Data Modeling, we encourage you to apply for this exciting opportunity.,