Senior Data Engineer role, Databricks

Hi,

We are hiring Senior Data engineer for our Client located at Chennai location. Hybrid mode.

Job Overview:

We are seeking an experienced Senior Data Engineer to join our dynamic team. The ideal candidate

will be responsible for Design and implement the data engineering framework.

Responsibilities

• Strong Skill in Big Query, GCP Cloud Data Fusion (for ETL/ELT) and PowerBI.
• Need to have strong skill in Data Pipelines
• Able to work with Power BI and Power BI Reporting
• Design and implement the data engineering framework and data pipelines using Databricks

and Azure Data Factory.

• Document the high-level design components of the Databricks data pipeline framework.
• Evaluate and document the current dependencies on the existing DEI toolset and agree a

migration plan.

• Lead on the Design and implementation an MVP Databricks framework.
• Document and agree an aligned set of standards to support the implementation of a

candidate pipeline under the new framework.

• Support integrating a test automation approach to the Databricks framework in conjunction

with the test engineering function to support

• CI/CD and automated testing.
• Support the development teams capability building by establishing an L&D and knowledge

transition approach.

• Support the implementation of data pipelines against the new framework in line with the

agreed migration plan.

• Ensure data quality management including profiling, cleansing and deduplication to support

build of data products for clients

Skills Knowledge and Expertise

Experience working in Azure Cloud using Azure SQL, Azure Databricks, Azure Data Lake, Delta

Lake, and Azure DevOps.

• Proficient in Python, Pyspark and SQL coding skills.

• Profiling data and data modelling experience on large data transformation projects creating

data products and data pipelines.

• Creating data management frameworks and data pipelines which are metadata and business

rules driven using Databricks.

• Experience of reviewing datasets for data products in terms of data quality management and

populating data schemas set by Data Modellers

• Experience with data profiling, data quality management and data cleansing tools.

Immediate joining or short notice is required.

Sal upto 35 lpa

Pls call Hemanth 9715166618 for more info

Thanks,

Hemanth

9715166618