Job
Description
About this opportunity: A&AI (SL IT & ADM) team is currently seeking a versatile and motivated DevOps Engineer (with expertise in Kubernetes and Cloud Infrastructure) to join the AI/ML team. This role will be pivotal in managing multiple platforms and systems, focusing on Kubernetes, ELK/Opensearch, and various DevOps tools to ensure seamless data flow for our machine learning and data science initiatives. The ideal candidate should have a strong foundation in Python programming, experience with Elasticsearch, Logstash, and Kibana (ELK), proficiency in MLOps, and expertise in machine learning model development and deployment. Additionally, familiarity with basic Spark concepts and visualization tools like Grafana and Kibana is desirable. What you will do: Design and implement robust AI/ML infrastructure using cloud services and Kubernetes to support machine learning operations (MLOps) and data processing workflows. Deploy, manage, and optimize Kubernetes clusters specifically tailored for AI/ML workloads, ensuring optimal resource allocation and scalability across different network configurations. Develop and maintain CI/CD pipelines tailored for continuous training and deployment of machine learning models, integrating tools like Kubeflow, MLflow, ArgoFlow or TensorFlow Extended (TFX). Collaborate with data scientists to oversee the deployment of machine learning models and set up monitoring systems to track their performance and health in production. Design and implement data pipelines for large-scale data ingestion, processing, and analytics essential for machine learning models, utilizing distributed storage and processing technologies such as Hadoop, Spark, and Kafka. . The skills you bring: Extensive experience with Kubernetes and cloud services (AWS, Azure, GCP, private cloud) with a focus on deploying and managing AI/ML environments. Strong proficiency in scripting and automation using languages like Python, Bash, or Perl. Experience with AI/ML tools and frameworks (TensorFlow, PyTorch, Scikit-learn) and MLOps tools (Kubeflow, MLflow, TFX). In-depth knowledge of data pipeline and workflow management tools, distributed data processing (Hadoop, Spark), and messaging systems (Kafka, RabbitMQ). Expertise in implementing CI/CD pipelines, infrastructure as code (IaC), and configuration management tools. Familiarity with security standards and data protection regulations relevant to AI/ML projects. Proven ability to design and maintain reliable and scalable infrastructure tailored for AI/ML workloads. Excellent analytical, problem-solving, and communication skills.
