Senior Data Engineer with Python & PySpark

Role Overview: At CGI, we are seeking a skilled and proactive Senior Software Engineer - Data Engineer with experience in Python, Kafka Streaming, and Spark. As part of the team, you will be responsible for designing, developing, and deploying scalable backend systems and real-time data pipelines in cloud-native environments. This role is ideal for individuals passionate about building high-performance systems with cutting-edge technologies. Key Responsibilities: - Develop real-time data streaming applications using Apache Kafka and Kafka Streams. - Build and optimize large-scale batch and stream processing pipelines with Apache Spark. - Containerize applications and manage deployments using Kubernetes. - Collaborate with DevOps teams to ensure robust and scalable CI/CD pipelines. - Write unit tests, conduct code reviews, and maintain code quality and reliability. - Troubleshoot and debug production issues across multiple environments. Qualifications Required To Be Successful In This Role: - Strong programming skills in Python. - Hands-on experience with Apache Kafka, Kafka Streams, and event-driven architecture. - Solid knowledge of Apache Spark for both batch and streaming processes. - Familiarity with microservices architecture, RESTful APIs, and distributed systems. - Experience with build tools like Maven or Gradle. - Proficiency in Git, Jenkins, CI/CD pipelines, and Agile development practices. - Excellent problem-solving skills and the ability to work in a fast-paced environment. Additional Company Details: CGI offers a competitive compensation package with opportunities for growth and professional development. Full-time, permanent members receive benefits starting on the first day, including a paid time-off program and profit participation and stock purchase plans. The company values ownership, teamwork, respect, and belonging, providing a supportive environment for employees to reach their full potential. Please note that only selected candidates will be contacted for interviews. No unsolicited agency referrals, please.,