Assistant Scientist- Data Engineering

Assistant Scientist -Data Engineering

Bangalore, Karnataka India & Gurgaon, Haryana-IndiaAXA XL recognizes data and information as critical business assets, both in terms of managing risk and enabling new business opportunities. This data should not only be high quality, but also actionable - enabling AXA XL's executive leadership team to maximize benefits and facilitate sustained dynamic advantage. Our Chief Data Office also known as our Innovation, Data Intelligence & Analytics team (IDA) is focused on driving innovation by optimizing how we leverage data to drive strategy and create a new business model - disrupting the insurance market.As we develop an enterprise-wide data and digital strategy that moves us toward a greater focus on the use of data and data-driven insights, we are seeking an Assistant Scientist for our Data Engineering team. The role will support the team's efforts towards creating, enhancing, and stabilizing the Enterprise data lake through the development of the data pipelines. This role requires a person that is a team player and can work well with team members from other disciplines to deliver data in an efficient and strategic manner.

DISCOVER your opportunity

What will your essential responsibilities include

• Design, Develop, and maintain ETL pipelines using Pyspark in Azure Databricks using delta tables.
• Create build from GitHub and release pipeline for Ingestion and Databricks using Azure DevOps / Harness
• Monitor the Performance of ETL Jobs, resolve any issues that arose, and improve the performance metrics as needed.
• Diagnose system performance issues related to data processing and implement the solution to address them.
• Collaborate with other teams to make sure successful integration of data pipelines into larger system architecture requirements.
• Maintain integrity and quality across all pipelines and environments.