Data Engineering Lead

7 - 10 years

Noida Gurugram Delhi

Posted:5 days ago

Job Description

Role & responsibilities

  • Data Modeling & Analytics Engineering
    • Design, develop, and maintain scalable dimensional data models, including Star and Snowflake schemas
    • Build curated data products that support reporting, analytics, AI, and machine learning use cases
    • Define and standardize business logic, KPIs, metrics, and calculations across enterprise domains
    • Create reusable datasets, semantic models, and data assets to improve consistency and reduce duplication
    • Collaborate with business stakeholders to translate requirements into analytical and reporting solutions
  • Databricks Platform Engineering
    • Design and implement modern data architectures leveraging Databricks Lakehouse principles
    • Develop and maintain Bronze, Silver, and Gold data layers
    • Build scalable ETL/ELT pipelines using Spark, PySpark, SQL, and Databricks Workflows
    • Optimize Delta Lake implementations using partitioning, Z-Ordering, data skipping, Change Data Feed (CDF)
  • Delta optimization techniques
    • Implement CI/CD pipelines and DevOps best practices for data engineering and analytics solutions
    • Context Layer & Semantic Layer Development
    • Design and manage enterprise semantic and context layers to provide a single source of truth for reporting and analytics
    • Standardize business metrics, dimensions, hierarchies, and definitions across reporting platforms
    • Enable self-service analytics through Power BI, Tableau, and AI-driven applications
    • Manage semantic models and governance processes to ensure reporting consistency
    • Establish metric certification, lineage tracking, and data governance standards
  • Performance Optimization & Cost Management
    • Analyze query execution plans and identify performance bottlenecks
    • Optimize Spark workloads for scalability, reliability, and cost efficiency
    • Implement cluster sizing, autoscaling, caching, broadcast joins, and Adaptive Query Execution (AQE) strategies
    • Tune Delta Lake tables through file compaction, partition optimization, and storage management
    • Monitor warehouse and cluster utilization to improve performance and control cloud expenses
    • Enhance dashboard and reporting performance through optimized data models and query design
  • Data Governance & Quality Management
    • Develop and implement data quality frameworks, validation processes, and automated monitoring
    • Ensure compliance with enterprise security, governance, and regulatory requirements
    • Support metadata management, data cataloging, and lineage initiatives
    • Establish monitoring and alerting for data pipelines, processing failures, and data anomalies
    • Partner with governance teams to drive data stewardship and certification processes
  • Technical Leadership & Collaboration
    • Collaborate with data scientists, analysts, engineers, product owners, and business leaders
    • Mentor and guide junior analytics engineers and data engineers
    • Conduct architecture reviews and recommend technology best practices
    • Drive adoption of enterprise data standards, frameworks, and governance processes
    • Lead technical design discussions and influence data platform strategy and roadmap
  • Comply with all applicable Company policies, procedures, and business directives, changes including those relating to work location, team assignments, work schedules, and flexible work arrangements

Preferred candidate profile

  • Master's degree in Computer Science, Data Science, Analytics, or a related discipline
  • Databricks certifications (Data Engineer Associate, Professional, or equivalent)
  • Experience with Databricks Unity Catalog and enterprise data governance implementations
  • Experience supporting AI/ML, Generative AI, or advanced analytics use cases
  • Experience designing enterprise semantic models and metric layers
  • Healthcare, insurance, or highly regulated industry experience
  • Demonstrated experience leading technical teams, mentoring engineers, and driving enterprise data strategy
  • Knowledge of data catalog and governance tools such as Purview, Collibra, or Alation

Key Competencies:

  • Data Architecture & Modeling
  • Analytics Engineering
  • Databricks & Delta Lake
  • Spark/PySpark Development
  • Data Governance & Quality
  • Performance Optimization
  • Semantic Layer Design
  • Cloud Data Platforms
  • Technical Leadership
  • Stakeholder Management

Skills Required

Data warehousing Data Visualization SQL Modeling Bi Data analysis Microsoft AWS

Learn the skills this job asks for

Free courses, interview questions and practice for each skill.

Work Mode

Hybrid

Job Type

Full Time

Mock Interview

Practice Video Interview with JobPe AI

Start Job-Specific Interview
cta

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.

coding practice

Enhance Your Skills

Practice coding challenges to boost your skills

Start Practicing Now
Optum logo
Optum

Hospitals and Health Care

Eden Prairie MN

RecommendedJobs for You

Altimetrik logo

Mumbai

The HEINEKEN Company logo

Data Engineering Lead

The HEINEKEN Company

Hyderabad

BOT Tech logo

Gurugram

Altimetrik logo

Mumbai

The HEINEKEN Company logo

Data Engineering Lead

The HEINEKEN Company

Hyderabad

BOT Tech logo

Gurugram