Data Engineer-Data Platforms-Azure-DLT

8+ Yrs total experience in IT/Data Engineering projects & 5+ years of

relevant experience- Strong coding skills in - Python, SQL, ADF, Databricks (PySpark, Delta

live table, Live/structural data streaming, Unity Catalog, Performance Tuning of DLT

pipelines with high volume of data processing , Cost Optimization of Databricks when DLT

runs in live streaming on Photon, Serverless, non serverless)The candidate should be

responsible, take ownership, self driven, dedication to achieve timeline. Should have great

problem solving and analytical/reasoning capability,Technical skill with good mindset and

behaviour is important good hands-on programming / scripting knowledge in Python , very

strong skilled resources only with good communication-knowledge in Power BI

AWS/Azure/SAP - Master

ELT - Master

Data Modelling - Master

Data Integration & Ingestion - Skill

Data Manipulation and Processing - Skill

GITHUB, Action, Azure DevOps - Skill

Data factory, Databricks, SQL DB, Synapse, Stream Analytics, Glue, Airflow, Kinesis,

Redshift, SonarQube, PyTest - Skill

Required skills-Python, SQL, Databricks, ADF within-Databricks - DLT, PySpark, Structural

streaming, performance and cost optimization

Azure : Azure data factory, ADLS- Azure data lake store, Azure data bricks,

Mandatory Programming languages: Py-Spark, PL/SQL, Spark SQL, Python Programming,

Data Modelling and design

Database : SQL DB

Experience with Azure: ADLS, Databricks, Stream Analytics, SQL DW, COSMOS DB,

Analysis Services, Azure Functions, Serverless Architecture, ARM Templates,

Experience with relational SQL and NoSQL databases, including Postgres and Cassandra.

Experience with object-oriented/object function scripting languages: Python, SQL, Scala,

Spark-SQL etc.

Data Warehousing experience with strong domain knowledge Nice to have experience with

big data tools: Hadoop, Spark, Kafka, etc.

Nice to have experience with data pipeline and workflow management tools: Azkaban, Luigi,

Airflow, stream processing systems: Storm, Azure Synapse/CI/CD

optional: COSMOS DB, Synapse Analytics, Azure analysis services