PySpark- Senior Developer

Job Description

Job Description Key responsibilities include the following: Develop and maintain scalable data pipelines using Pyspark and proven experience as developer with expertise in PySpark. Good to have knowledge on Ab Initio. Experience with distributed computing and parallel processing. Proficiency in SQL and experience with database systems. Collaborate with data engineers and data scientists to understand and fulfil data processing needs. Optimize and troubleshoot existing PySpark applications for performance improvements. Write clean, efficient, and well-documented code following best practices. Participate in design and code reviews. Develop and implement ETL processes to extract, transform, and load data. Ensure data integrity and quality throughout the data lifecycle. Stay current with the latest industry trends and technologies in big data and cloud computing Show more Show less