Big Data Engineer

Job Title: Python Databricks Engineer

Experience: 7–10 Years

Location: Noida

Job Summary

We are seeking a highly skilled Python Databricks Engineer with strong expertise in data engineering and cloud technologies. The ideal candidate will have a solid background in building scalable data pipelines, working with Databricks/Spark, and leveraging AWS services for data processing. Experience in healthcare data systems is a plus.

Key Responsibilities-

Design, develop, and optimize scalable data pipelines and workflows using Databricks, Spark, and Python.

Work with AWS services (S3, Lambda) to integrate and process large-scale datasets.

Develop, schedule, and monitor ETL/ELT processes using Airflow and other orchestration tools.

Collaborate with cross-functional teams to ensure data quality, security, and governance.

Requirements-

7–10 years of overall IT experience with at least 5+ years in Data Engineering.

2+ years of hands-on Python development.

Strong experience with Databricks and Spark.

Proficiency in SQL for querying and transforming data.

Hands-on experience with AWS technologies (S3, Lambda).

Experience with DataStage or similar ETL tools.

Strong knowledge of Airflow for workflow orchestration.

Exposure to healthcare data processing systems is highly desirable.

Ability to work in hybrid teams, mentor juniors, and provide technical guidance.

Technical Stack-

Databricks, Python, Spark, SQL

AWS (S3, Lambda)

Healthcare Data Processing Systems

Batch Processing Frameworks

Airflow, DataStage (or similar ETL tools)

Job Type: Full-time

Pay: ₹166,000.00 - ₹190,000.00 per month

Work Location: In person