Sr. Data Engineer

As an ETL Developer at our company, you will be responsible for designing, developing, and implementing performant ETL pipelines using Python API (pySpark) of Apache Spark on AWS EMR. Your role will involve the following key responsibilities: - Programming with Python with a strong proficiency in Python - Utilizing functional programming concepts and Data Modelling - Hands-on experience in developing ETL data pipelines using pySpark on AWS EMR - Understanding Sparks Data frame and API - Configuring EMR clusters on AWS - Dealing with AWS S3 object storage from Spark - Troubleshooting Spark jobs and monitoring them using Spark UI Qualifications required for this role include: - 5+ years of experience in programming with Python - 3+ years of hands-on experience in developing ETL data pipelines using pySpark on AWS EMR - Familiarity with functional programming concepts and Data Modelling - Good understanding of Sparks Data frame and API - Experience in configuring EMR clusters on AWS - Experience in dealing with AWS S3 object storage from Spark - Experience in troubleshooting Spark jobs and knowledge of monitoring Spark jobs using Spark UI This is a Full-time, Permanent job opportunity. The benefits include the flexibility of working from home. The work schedule consists of Day shift, Fixed shift, and Morning shift. The preferred experience levels for various skills are: - ETL: 6 years - Python: 6 years - Apache Spark: 6 years - pySpark: 7 years - AWS S3: 6 years - Data Modelling: 6 years The work location for this position is in person. The application deadline is 22/02/2025.,