Jobaaj Com logo

Senior Data Engineer

Jobaaj Com

5 - 8 years

12 - 15 Lacs

Noida

Posted:4 hours ago

Job Description

Purpose of the Job

We are looking for a Senior Data Engineer who is equally strong in data modelling, data quality, SQL, and business understanding, as well as engineering skills such as Python, PySpark, CI/CD, performance optimization, and reliability.

The candidate will be responsible for designing and delivering production-ready data platforms, from gathering requirements to deployment and production support. The role also involves working directly with clients and sharing technical knowledge with the wider engineering team.

Technology Stack:

Key Responsibilities

  1. Own the end-to-end design and delivery of data pipelines and platforms, including requirements gathering, architecture, development, testing, deployment, and production support.
  2. Design data models, including dimensional modelling, SCD, and Medallion Architecture (Bronze/Silver/Gold), based on business requirements.
  3. Build and maintain scalable ETL/ELT pipelines using Python, SQL, PySpark, and Azure Data Factory on Azure Databricks or Microsoft Fabric.
  4. Implement batch, incremental/CDC, and streaming data loads.
  5. Optimize Spark and SQL performance, ensuring scalability and cost efficiency.
  6. Implement data quality checks, monitoring, and alerting across pipelines.
  7. Apply data governance, lineage, and access control practices.
  8. Automate testing and deployment using Git and CI/CD while following engineering best practices.
  9. Work directly with client stakeholders to gather requirements, explain technical design decisions, and establish acceptance criteria.
  10. Integrate data from APIs, transactional systems, and external platforms.
  11. Investigate and resolve complex production issues and document their root causes.
  12. Maintain technical documentation and contribute to team knowledge-sharing through peer reviews and technical discussions.

Essential Qualifications

  • Bachelor's degree in any discipline, preferably in Computer Science, Information Systems, or a related field.
  • 58 years of relevant experience in Data Engineering.

Required Skills and Qualifications

Data Engineering Skills:

  • Strong SQL knowledge.
  • Experience in dimensional modelling and Medallion/Lakehouse architecture.
  • Understanding of data quality practices.
  • Ability to translate business requirements into effective data solutions.

Technical and Engineering Skills:

  • Strong hands-on experience with Python and PySpark.
  • Knowledge of Git, CI/CD, and automated testing.
  • Experience in performance tuning of high-volume workloads.
  • Hands-on experience with Azure Databricks or Microsoft Fabric.
  • Experience with Azure Data Factory or an equivalent orchestration tool.
  • Proven experience in developing batch and incremental/CDC pipelines using Delta Lake or similar technologies.

Client Management Skills:

  • Experience in gathering client requirements.
  • Strong stakeholder communication and collaboration skills.
  • Ability to prepare clear technical documentation.
  • Ability to demonstrate measurable improvements in runtime, cost, or data quality.

Good to Have

  • Experience with streaming technologies such as Spark Structured Streaming and Kafka.
  • Knowledge of governance and lineage tools such as Unity Catalog and Azure Purview.
  • Understanding of Power BI and semantic-layer concepts.
  • Microsoft certifications such as DP-203 or DP-600.
  • Experience with Infrastructure as Code (Terraform).

Benefits and Additional Information

  • Medical insurance coverage for employees.
  • Provident Fund (PF) facility.
  • Quarterly team parties and annual outings/trips.
  • Regular leadership check-ins for professional growth and feedback.
  • Recognition awards for high performance.
  • Team engagement activities and events throughout the year.

Skills Required

Senior SQL Data modeling Factory Pyspark Development Data Fabric

Learn the skills this job asks for

See your skill gap for Data Engineer →

Free courses, interview questions and practice for each skill.

Work Mode

Work from Office

Job Type

Full Time

Mock Interview

Practice Video Interview with JobPe AI

Start PySpark Interview
cta

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.

coding practice

Enhance Your Skills

Practice coding challenges to boost your skills

Start Practicing Now
Jobaaj Com logo
Jobaaj Com

RecommendedJobs for You

Recruito logo

India

Jobaaj Com logo

Noida

YouGov logo
HP logo
Ecolab logo

Bengaluru

Hiringhood logo

Hyderabad

IntraEdge logo

Pune

NationsBenefits logo

Senior Data Engineer

NationsBenefits

Hyderabad

P

Hyderabad