Senior Python Pyspark Developer

You are a skilled and proactive Python / PySpark Developer sought to join the data engineering or analytics team. Your responsibilities will include building scalable data pipelines, performing large-scale data processing, and collaborating with data scientists, analysts, and business stakeholders. You will design, develop, and optimize ETL data pipelines using PySpark on big data platforms (e.g., Hadoop, Databricks, EMR). Writing clean, efficient, and modular code in Python for data processing and integration tasks is essential. Working with large datasets to extract insights, transform raw data, and ensure data quality will be part of your daily tasks. Collaborating with cross-functional teams to understand business requirements and translating them into technical solutions is crucial. You will also need to implement performance tuning and debugging of PySpark jobs, monitor and troubleshoot data workflows and batch jobs in production environments, and document solutions while maintaining code repositories (e.g., Git). Required qualifications include proficiency in Python with experience in building data-centric applications, strong experience with PySpark, understanding of Spark internals, hands-on experience with Hadoop ecosystem, Hive, or cloud-based big data platforms like AWS EMR, Azure Databricks, or GCP DataProc, familiarity with workflow orchestration tools, good understanding of SQL and relational databases, experience with version control systems, strong problem-solving skills, and a Bachelor's degree in Computer Science, Engineering, or a related field. Preferred qualifications include experience with CI/CD pipelines and DevOps practices, knowledge of data warehousing and data modeling, exposure to streaming technologies like Kafka and Spark Streaming, and familiarity with containerization tools like Docker or Kubernetes. At Virtusa, teamwork, quality of life, professional and personal development are valued. Joining a global team of 27,000 people, you will have exciting projects, opportunities, and work with state-of-the-art technologies throughout your career. Collaboration and a dynamic team environment are key aspects of Virtusa, providing an environment to nurture new ideas and foster excellence.,