Python developer with Spark

Role Overview: As a skilled developer, you will be responsible for developing and maintaining scalable data pipelines utilizing Python and PySpark. Your role involves collaborating closely with data engineers and data scientists to comprehend and meet data processing requirements efficiently. Your tasks will also include optimizing and resolving issues in existing PySpark applications to enhance performance. It is essential to write clean, efficient, and well-documented code adhering to industry best practices. Furthermore, you will actively participate in design and code reviews to ensure quality and consistency across projects. Additionally, you will be involved in developing and executing ETL processes for data extraction, transformation, and loading purposes. Overall, your contribution will be vital in ensuring the seamless operation and enhancement of data processing capabilities within the organization. Key Responsibilities: - Develop and maintain scalable data pipelines using Python and PySpark - Collaborate with data engineers and data scientists to meet data processing requirements efficiently - Optimize and resolve issues in existing PySpark applications to enhance performance - Write clean, efficient, and well-documented code following industry best practices - Participate in design and code reviews to ensure quality and consistency - Develop and execute ETL processes for data extraction, transformation, and loading purposes Qualifications Required: - Proficiency in Python and PySpark - Experience in developing data pipelines - Strong problem-solving skills - Ability to work collaboratively in a team environment - Familiarity with ETL processes and best practices,