SAN Engineering Solutions logo

Big Data Developer

SAN Engineering Solutions

4 - 15 years

Bengaluru

Posted:5 days ago

Job Description

WE ARE HIRING | BIG DATA ENGINEERS

SAN Engineering Solutions is expanding its Big Data Engineering team and looking for talented professionals with strong expertise in Python, Spark, PySpark, SQL, and Big Data platforms.


Explore an exciting opportunity to work on large-scale data migration projects involving Apache Iceberg!


Job Details

  • Role:

    Big Data Engineer
  • Experience:

    4–15 Years
  • Locations:

    Chennai – Sholinganallur | Bangalore – RMZ Ecoworld


Experience-Wise Hiring

🔹 4–7 Years – 10 Positions

🔹 7–10 Years – 10 Positions

🔹 10–15 Years – 10 Positions


Mandatory Skills

  • Python
  • Apache Spark
  • PySpark
  • SQL
  • Unix Shell Scripting
  • Big Data Platforms


Good to Have Skills

  • Apache Airflow
  • Data Warehousing
  • Data Modelling
  • Apache Iceberg Development


Key Responsibilities

✔️ Design and develop scalable PySpark-based data migration pipelines from source systems to Apache Iceberg.

✔️ Perform data extraction, transformation, cleansing, standardization, and loading.

✔️ Develop complex SQL queries and perform query tuning and performance optimization.

✔️ Analyze Spark execution plans and optimize jobs using partitioning, caching, and efficient transformations.

✔️ Perform data profiling, validation, and reconciliation to ensure data quality and integrity.

✔️ Troubleshoot PySpark jobs, production issues, and performance bottlenecks.

✔️ Monitor Azure DevOps (ADO) pipelines and manage code changes using Git, pull requests, and code reviews.

✔️ Conduct RCA, support deployments, SIT, UAT, hypercare, and production support.

✔️ Collaborate with cross-functional teams to ensure successful project delivery.



#WeAreHiring #BigDataEngineer #BigData #DataEngineering #Python #PySpark #ApacheSpark #SQL #ApacheIceberg #Airflow #DataMigration #DataWarehousing #DataModelling #AzureDevOps #ChennaiHiring #BangaloreHiring #ITJobs #TechHiring #HiringNow

Skills Required

BIG Data Python Spark Pyspark SQL Data Migration Apache Apache Spark Unix Shell Scripting Data Warehousing Data Modelling Data Extraction

Job Type

Full Time

Mock Interview

Practice Video Interview with JobPe AI

Start Big Data Interview
cta

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.

coding practice

Enhance Your Python Skills

Practice Python coding challenges to boost your skills

Start Practicing Python Now
SAN Engineering Solutions logo
SAN Engineering Solutions

RecommendedJobs for You

SAN Engineering Solutions logo

Big Data Developer

SAN Engineering Solutions

Bengaluru

SAN Engineering Solutions logo

Big Data Developer

SAN Engineering Solutions

Bengaluru