Staff / Principal Data & MLOPs Engineer

Job Description

Promote DataOps approach to Data science, engineering and analytics delivery processes to automate the provision of data, testing and monitoring and shorten CI/CD. Collaborate with data ML leads and create and build optimal data pipeline architecture for the data solutions including data science products Ensure the data pipelines are scalable and performant as we'll as creating and maintaining service to connect data products Create dashboards and other tools required to efficiently monitor our data and ML infrastructure, pipelines, ETL and analytics delivery processes. Building end-to-end event instrumentation and alerting system to detect and alert any anomaly in the system or in the data Assist in managing our data and ML infrastructure (upgrading, monitoring, optimising) Collaborate with IT DevOps engineers and participate in enterprise DevOps activities. Exchange your knowledge on infra and data standards with other developers and be part of our tech community. Promote the use of engineering best practices. Contribute to innovative POCs with our data engineering teams. Remain flexible towards technology approaches to ensure that the best advantage is being taken by new technologies. Required skills Qualifications: Strong drive to solve problems, communicate clearly and contribute positively to a DevOps/DataOps culture Knowledge of the latest DevOps tools and practices. Experience with data pipelines within AWS (Glue, DataPipeline, Athena, EMR, DMS, Spark) Experience of Database Replication and databases like Aurora, MySQL, MariaDB, etc Efficient in building CI/CD pipelines for containerized Java/Python codestack Comfortable with Git workflow. Experience with applications deployed in AWS. Experience with configuration management and provisioning tools (eg, Ansible, CloudFormation, Terraform) Knowledge of one or more scripting languages - Bash/Python/JavaScript Orchestration/containerisation using Docker and Kubernetes. Basic knowledge of data science ML engineering Bachelors Degree in computer science or similar degree or Big Data Background from top tier universities.