Senior Data Engineer

Level:

Location:

LOS:

Competency:

Skill: SQL, Pyspark, Databricks and Python

Job Position Title:

Senior Data Engineer with Python Programming Senior Consultant/Manager (8+ Years)

Responsibilities:

• Design, develop, and manage scalable and secure data pipelines using Databricks and PySpark.
• Architect and implement cloud-based data solutions, integrating structured and unstructured data sources.
• Build and optimize ETL workflows and ensure seamless data integration across platforms.
• Develop data models using normalization and denormalization techniques to support OLTP and OLAP systems.
• Troubleshoot performance bottlenecks in data flows and ETL processes.
• Integrate advanced analytics and support BI use cases within the Cloud ecosystem.
• Lead code reviews and ensure adherence to version control practices (e.g., Git).
• Contribute to the design and deployment of enterprise-level data warehousing solutions.
• Stay current with Cloud technologies and Python ecosystem updates to adopt best practices and emerging tools.

Mandatory skill sets:

• Strong SQL skills (Must-Have)

• Databricks (Must-Have)

• Python programming (Must-Have)

• Spark (Must-Have)

• Data modeling (Normalization & Denormalization)

• Data warehousing and BI tools integration
• Git (Version Control)
• Building scalable ETL pipelines

Preferred skill sets (Good to Have):

• Understanding of OLTP and OLAP environments
• Experience with Kafka and Hadoop
• Azure Cloud knowledge
• Azure DevOps for CI/CD integration
• Agile delivery methodologies

Years of experience required:

• 8+ years of overall experience in cloud engineering or data engineering roles, with at least 2-3 years of hands-on experience with cloud services.

Education qualification:

BE/B.Tech/MBA/MCA