Senior DevOps Engineer (BigData Python)

Our client is a global commerce leader where you can influence how the world buys, sells, and gives. Youll be part of a work culture thats been genuinely committed to diversity and inclusion since its founding over twenty five years ago. Here, you can be yourself, do your
best work along with a team of professionals, and have a meaningful impact on people across the globe. We seek people with drive, ideas, and a passion for helping small businesses succeed to help.

About the team:

We are the AI Platform Team, providing highly available, scalable, and automated machine learning and big data infrastructure for researchers and data scientists globally. We are looking for a motivated, self-reliant SRE / DevOps engineer with Java and Python experience to drive operational excellence, automation, and platform reliability.

Role Overview:

This role focuses on maintaining, deploying, and improving Big Data and AI platform services with strong emphasis on DevOps, SRE practices, and automation. You will work closely with developers, researchers, and infrastructure teams to ensure robust, scalable, and highly available distributed data pipelines.

Key Responsibilities (DevOps-heavy, ~60%):

• Design, implement, and maintain CI/CD pipelines for Big Data and AI platform applications.
• Manage and troubleshoot Kubernetes clusters, Docker containers, and cloud infrastructure.
• Ensure high availability (99.999%), system reliability, and security across platforms.
• Automate operational tasks, monitoring, and deployment workflows.
• Collaborate with developers and data engineers to deploy and scale distributed pipelines efficiently.
• Analyze and resolve production issues, performance bottlenecks, and functional problems.
• Define operational standards, versioning practices, and advise teams on DevOps best practices.
• Prepare documentation, training materials, and provide technical support to platform users.
  

Development Responsibilities (~40%):

• Design, build, and refactor services in Java and Python for Big Data pipelines.
• Work with Big Data technologies such as Hadoop, Spark, Flink, Hive, and HDFS.
• Handle data integration, bug fixing, performance optimization, and feature enhancements.
• Build batch pipelines to support billion-scale embedding generation for text, image, and video (plus).
• Work with vector similarity search, vector databases, and data quality/governance processes.
  

Required Skills & Experience:

• Strong Java and Python development experience (2-4 years).
• Hands-on experience with Big Data technologies: Hadoop, Spark, Flink, Hive, HDFS.
• Experience with CI/CD pipelines (Jenkins or similar), test automation, and DevOps practices.
• Solid understanding of Kubernetes, Docker, Linux fundamentals, and monitoring tools.
• Strong debugging and triaging skills for distributed systems.
• Excellent communication and collaboration skills with cross-functional teams.
• Strong organizational skills to manage multiple projects in a fast-paced environment.
• Fluent in English (spoken and written).