Home
Jobs

Senior Data Engineer

5 years

20 - 25 Lacs

Posted:1 day ago| Platform: Linkedin logo

Apply

Work Mode

On-site

Job Type

Contractual

Job Description

Job Title:

Data Engineer – AWS, PySpark, Databricks

Location: Mumbai (Bandra kurla complex )

Experience Required:

5+ Years

Employment Type:

Contract

Job Overview

We are seeking a

highly skilled Data Engineer

with strong hands-on experience in

AWS cloud services

,

PySpark

, and

Databricks

to join our advanced analytics and data platform team. The ideal candidate should have expertise in building scalable data pipelines, transforming large datasets, and managing cloud-based big data environments. You’ll work closely with data scientists, architects, and business teams to deliver efficient data solutions.

Key Responsibilities

  • Design, develop, and maintain robust ETL/ELT pipelines using PySpark on Databricks.
  • Work with structured and semi-structured data on large-scale data platforms.
  • Integrate data from various sources into the AWS data lake and Delta Lake.
  • Optimize Spark jobs for performance, cost, and reliability.
  • Collaborate with business stakeholders to understand data requirements and translate them into scalable solutions.
  • Use AWS services (S3, Glue, Lambda, EMR, Redshift, Athena) to manage data pipelines and orchestrations.
  • Write clean, maintainable, and reusable PySpark code with proper logging and error handling.
  • Implement data governance, quality checks, and documentation practices.
  • Perform data validation and reconciliation to ensure accuracy and consistency.
  • Participate in code reviews, unit testing, and continuous integration/delivery processes.

Required Skills & Qualifications

  • 5+ years of experience in data engineering or big data development.
  • Minimum 3+ years of experience working with PySpark and Apache Spark.
  • Strong experience working on Databricks (Notebooks, Workflows, Delta Lake).
  • Proficiency with AWS services: S3, Lambda, Glue, EMR, Redshift, and Athena.
  • Expertise in working with large-scale data ingestion, transformation, and storage.
  • Solid understanding of data partitioning, file formats (Parquet, ORC, JSON), and performance tuning in Spark.
  • Experience in CI/CD practices, version control (Git), and deployment automation.
  • Familiarity with Airflow, DBT, or other workflow orchestration tools is a plus.
  • Strong communication and collaboration skills in agile environments.

Good To Have

  • Experience with Delta Lake architecture and advanced Databricks features.
  • Knowledge of SQL optimization and data warehousing concepts.
  • Experience integrating with BI tools like Power BI or Tableau.
  • Exposure to machine learning pipelines or integration with MLflow.
  • AWS certifications (e.g., AWS Certified Data Analytics, Solutions Architect).
Skills: data governance,data quality,athena,aws,databricks,git,python,emr,lambda,dbt,etl,redshift,sql,airflow,ci/cd,data warehousing,pyspark,elt,glue,s3,spark

Mock Interview

Practice Video Interview with JobPe AI

Start Technical Interview
cta

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.

Job Application AI Bot

Job Application AI Bot

Apply to 20+ Portals in one click

Download Now

Download the Mobile App

Instantly access job listings, apply easily, and track applications.

coding practice

Enhance Your Python Skills

Practice Python coding challenges to boost your skills

Start Practicing Python Now
NSKH Tech

23 Jobs

RecommendedJobs for You

Kolkata, Mumbai, New Delhi, Hyderabad, Pune, Chennai, Bengaluru

Hyderabad, Pune, Bengaluru

Bengaluru, Karnataka, India