Python and PySpark Developer

You have an exciting opportunity to join a dynamic and high-impact firm as a Python, PySpark, and SQL Developer with 8-12 years of relevant experience. In this role, you will be responsible for working on development activities, collaborating with cross-functional teams, and designing scalable data pipelines using Python and PySpark. You will also be involved in implementing ETL processes, developing Power BI reports and dashboards, and optimizing data pipelines for performance and reliability. The ideal candidate should have 8+ years of experience in Spark, Scala, and PySpark for big data processing. Proficiency in Python programming for data manipulation and analysis is essential, along with knowledge of Python libraries such as Pandas and NumPy. Strong knowledge of SQL for querying databases and experience with database systems like Lakehouse, PostgreSQL, Teradata, and SQL Server are also required. Additionally, candidates should have strong analytical and problem-solving skills, effective communication skills, and the ability to troubleshoot and resolve data-related issues. Key Responsibilities: - Work on development activities and lead activities - Coordinate with Product Manager and Development Architect - Collaborate with other teams to understand data requirements and deliver solutions - Design, develop, and maintain scalable data pipelines using Python and PySpark - Utilize PySpark and Spark scripting for data processing and analysis - Implement ETL processes to ensure accurate data processing and storage - Develop and maintain Power BI reports and dashboards - Optimize data pipelines for performance and reliability - Integrate data from various sources into centralized data repositories - Ensure data quality and consistency across different data sets - Analyze large data sets to identify trends, patterns, and insights - Optimize PySpark applications for better performance and scalability - Continuously improve data processing workflows and infrastructure If you meet the qualifications and are interested in this incredible opportunity, please share your updated resume along with total experience, relevant experience in Python, PySpark, and SQL, current location, current CTC, expected CTC, and notice period. We assure you that your profile will be handled with strict confidentiality. Apply now and be a part of this amazing journey! Thank you, Syed Mohammad syed.m@anlage.co.in,