Airflow Azure DevOps Engineer

Job Description

Data Technology group in MSCI is responsible to build and maintain state-of-the-art data management platform that delivers Reference. Market & other critical datapoints to various products of the firm. The platform, hosted on firms data centers and Azure & GCP public cloud, processes 100 TB+ data and is expected to run 24*7. With increased focus on automation around systems development and operations, Data Science based quality control and cloud migration, several tech stack modernization initiatives are currently in progress. To accomplish these initiatives, we are seeking a highly motivated and innovative individual to join the automation team for the purpose of supporting our next generation of developer tools and infrastructure. The team is the hub around which Engineering, and Operations team revolves for automation and is committed to provide self-serve tools to our internal customers. The position is based in Mumbai, India office. Your Key Responsibilities Deploy and manage Airflow infrastructure . Monitor Airflow DAG execution and performance. Build and maintain Azure, GCP & on-prem datacenter infrastructure Build and maintain Azure DevOps CD / CI pipelines Automate Terraform deployment using Azure DevOps Implement actionable monitoring solution that can assist in building highly observable systems Provide support and continuous feedback to developers Work closely with development team to implement new build processes and strategies to meet new product requirements Independently engage with stakeholders and finalize requirements and bring them to a closure in a timely and effective manner. Address vulnerabilities in both private and public cloud infrastructure, with a focus on application vulnerabilities (Mend) and infrastructure (Wiz). Collaborate with the central DevOps teams to design and develop CI/CD pipelines and processes in accordance with MSCI policies and standards. Your skills and experience that will help you excel Self-motivated, collaborative individual with passion for excellence E Computer Science or equivalent with 5+ years of total experience and at least 2 years of experience in working with Azure DevOps tools and technologies Good working knowledge of source control applications like git with prior experience of building deployment workflows using this tool Good working knowledge of YAML, Python, Bash, Terraform Debugging maven builds Setting up YAML based CD/CI pipeline on Azure Containerizing applications using Docker Kubernetes administration, architecture and deployments Linux administration and scripting Helm charts Experience with creation, maintenance and deployment to the below resources Kubernetes (Rancher, Azure Kubernetes Services, Google Kubernetes Engine) Azure & GCP public cloud infrastructure Experience of working with Airflow is a big plus Experience with PostgresSQL is an added advantage