Data Engineer || 6 to 12 years

Hello Greetings !!

we are hiring for Data Engineer for one of the MNC company with INNOVA.

Senior Data Engineer

Responsibilities:

• Design, develop, and maintain scalable data pipelines using Python and modern data engineering tools.
• Understanding of banking data models, financial reporting, and regulatory compliance.
• Integrate data from legacy banking platforms into modern data lakes and warehouses.
• Collaborate with business analysts, architects, and operations teams to understand core banking workflows and data requirements.
• Design, build, and maintain scalable and reliable data pipelines using Azure Data Factory, Databricks, Synapse (Azure SQL DW), and Azure Data Lake.
• Perform data analysis and transformation using PySpark on Azure Databricks or Apache Spark.
• Design and implement data models optimized for storage and various query patterns.
• Work with structured, semi-structured, and unstructured data.
• Utilize various database technologies including Traditional RDBMS, MPP, and NoSQL.
• Ensure compliant handling and management of data.
• Experience in building end-to-end data pipelines in a cloud environment.
• Mentor junior engineers and contribute to the development of best practices and reusable frameworks.

Skills Expected:

• 8+ years of relevant IT experience in the BI/DW domain with hands-on experience on the

  Azure

  modern data platform.
• Strong knowledge and experience with the Python programming language
• Experience using version control systems like Git, bitbucket.
• Experience in data analysis and transformation using PySpark on Azure Databricks or Apache Spark.
• Good knowledge of Distributed Processing using Databricks or Apache Spark.
• Experience in creating data structures optimized for storage and various query patterns.
• Experience of integrating Unity Data Catalog with the Databricks and registering data assets etc.
• Understanding of CI/CD pipelines.
• Knowledge of Agile/Scrum methodologies.