Python Pyspark Developer

As a Python Developer, your role involves designing, developing, and implementing performant ETL pipelines using pySpark on Apache Spark within AWS EMR. Your responsibilities will include: - Writing reusable, testable, and efficient code for ETL pipelines - Integrating data storage solutions in Spark, particularly with AWS S3 object storage - Performance tuning of pySpark scripts to ensure optimal performance - Ensuring overall build delivery quality is of high standard and all deliveries are made on time - Handling customer meetings effectively with excellent communication skills Qualifications required for this role include: - Minimum 5 years of experience in programming with Python, demonstrating strong proficiency - Familiarity with functional programming concepts - At least 3 years of hands-on experience in developing ETL data pipelines using pySpark on AWS EMR - Hands-on experience with JSON processing using Python - Good understanding of Spark's RDD API and Dataframe API - Experience in configuring EMR clusters on AWS and working with Apache Spark Data sources API - Experience with troubleshooting and performance tuning of Spark jobs - Understanding of fundamental design principles behind business processes Nice to have skills include: - Knowledge of AWS SDK CLI - Experience setting up continuous integration/deployment of Spark jobs to EMR clusters - Familiarity with scheduling Spark applications in AWS EMR cluster - Understanding the differences between Hadoop MapReduce and Apache Spark - Proficient understanding of code versioning tools such as Git and SVN Experience required for this role is a minimum of 4-7 years. As a Python Developer, your role involves designing, developing, and implementing performant ETL pipelines using pySpark on Apache Spark within AWS EMR. Your responsibilities will include: - Writing reusable, testable, and efficient code for ETL pipelines - Integrating data storage solutions in Spark, particularly with AWS S3 object storage - Performance tuning of pySpark scripts to ensure optimal performance - Ensuring overall build delivery quality is of high standard and all deliveries are made on time - Handling customer meetings effectively with excellent communication skills Qualifications required for this role include: - Minimum 5 years of experience in programming with Python, demonstrating strong proficiency - Familiarity with functional programming concepts - At least 3 years of hands-on experience in developing ETL data pipelines using pySpark on AWS EMR - Hands-on experience with JSON processing using Python - Good understanding of Spark's RDD API and Dataframe API - Experience in configuring EMR clusters on AWS and working with Apache Spark Data sources API - Experience with troubleshooting and performance tuning of Spark jobs - Understanding of fundamental design principles behind business processes Nice to have skills include: - Knowledge of AWS SDK CLI - Experience setting up continuous integration/deployment of Spark jobs to EMR clusters - Familiarity with scheduling Spark applications in AWS EMR cluster - Understanding the differences between Hadoop MapReduce and Apache Spark - Proficient understanding of code versioning tools such as Git and SVN Experience required for this role is a minimum of 4-7 years.