Senior Technical Lead

India

Posted:1 day ago

Job Description

Job Summary

Hands-on build (the core of the role)

  • Design and build data pipelines in Microsoft Fabric — Data Factory pipelines, Copy activities, and Spark notebooks (PySpark) — across a bronze/silver/gold medallion architecture.
  • Write production-grade PySpark and SQL for ingestion, transformation, cleansing, and star-schema modelling in Delta Lake / OneLake.
  • Build and optimise semantic models (Direct Lake), including relationships, DAX measures, and row-level security, tuned for performance against the gold layer.
  • Design and implement metadata-driven ingestion frameworks — config-driven, parameterised pipelines that scale across many sources rather than one pipeline per table.
  • Implement robust handling for incremental loading, schema drift/evolution, and data quality, including CDC/watermark/hash-based patterns as appropriate per source.
  • Integrate and configure governance via Microsoft Purview — lineage, cataloguing, classification — as a first-class part of the build, not an afterthought.

Technical leadership

  • Own the technical architecture and set the design patterns, coding standards, and best practices the team builds against.
  • Lead code reviews, establish CI/CD for Fabric (Git integration, deployment pipelines, environment/workspace strategy), and enforce version control discipline.
  • Make and document the key build-vs-buy and architectural trade-off decisions (lakehouse vs. warehouse, batch vs. streaming, load strategies).
  • Partner with data governance, infrastructure, and security on workspace design, access, networking (e.g. Private Endpoints), and capacity/cost management.

Delivery & mentorship

  • Translate business and analytical requirements into a scoped, sequenced technical roadmap.
  • Mentor and grow data engineers — pairing, design guidance, and raising the team's Fabric capability.
  • Manage delivery within a sprint/Agile cadence (Azure DevOps), balancing hands-on contribution with unblocking the team.
  • Communicate status, risks, and trade-offs clearly to technical and non-technical stakeholders.
Key Responsibilities
  • Strong hands-on experience building data platforms on Microsoft Fabric (or very recent Azure Synapse / Databricks + Power BI experience directly transferable to Fabric).
  • Expert PySpark and strong SQL; production experience with Delta Lake and the medallion architecture.
  • Proven experience designing and building data pipelines — both orchestration (Data Factory / pipelines) and transformation (Spark notebooks) — and knowing when to use which.
  • Hands-on semantic modelling and DAX, ideally with Direct Lake, including performance optimisation.
  • Solid grasp of data warehousing and dimensional modelling (star schema, SCD, surrogate keys).
  • Experience with incremental load patterns, schema drift handling, and data quality frameworks.
  • CI/CD and Git for data platforms; workspace/environment management.
  • A track record of technical leadership — setting standards, reviewing code, mentoring — while remaining hands-on.
Skill Requirements
Must Have Skill
PySpark,Python,Microsoft Fabric
Good to Have Skill
Other Requirements

Skills Required

Spark Pyspark SQL Data Quality Technical Leadership Technical Architecture Design Patterns Ci/cd GIT Version Control Data Governance Networking

Learn the skills this job asks for

See your skill gap for Software Developer →

Free courses, interview questions and practice for each skill.

Mock Interview

Practice Video Interview with JobPe AI

Start PySpark Interview
cta

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.

coding practice

Enhance Your Skills

Practice coding challenges to boost your skills

Start Practicing Now
HCLTech logo
HCLTech

Information Technology Services

New Delhi

RecommendedJobs for You

infinite Computer Solution logo

Senior Technical Lead

infinite Computer Solution

India

infinite Computer Solution logo

Senior Technical Lead

infinite Computer Solution

Pune