Sr. Data Engineer

NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.

We are currently seeking a Sr. Data Engineer to join our team in Bangalore, Karn?taka (IN-KA), India (IN).

Primary Responsibility as Data Engineer

1) Build and implement PySpark-based data pipelines in Azure Synapse to transform and load data into ADLS in Delta format

2) Design and implement dimensional (star/snowflake) and 3NF data models. Optimized for access using Power BI

3) Perform unit testing of data pipelines and its transformations

4) Design and build metadata driven data pipelines using Pyspark in Azure Synapse

5) Analyze and optimize Spark SQL queries

6) Optimize integration of data lake withPower BI semantic model

7) Collaborate with cross-functional teams (data architects, engineers, and POs) to ensure data models align with business needs

8) Perform STM(source to target mapping) from source to multiple layers in the data lake

9) Maintain version control and CI/CD pipelines in Git and Azure DevOps

10) Integrate Azure Purview to enable access controls. Additionally implement row level security

7+ Years of experience in : SQL, PySpark

Hands-on experience with Azure Synapse, ADLS, Delta format, and metadata-driven data pipelines

Experienced in implementing dimensional (star/snowflake) and 3NF data models

Experienced in PySpark and Spark SQL, including query optimization and performance tuning

Experienced in writing complex SQL including windowing functions

Experienced in performing Source-to-Target Mapping (STM) by analyzing source systems and defining transformation logic

Strong problem-solving and analytical skills for debugging and optimizing data pipelines in Azure Synapse

Familiarity with CI/CD practices in Git and Azure DevOps

Working experience in Azure DevOps-based development environment