Data Engineer. -Databricks, PySpark, and Python

Job Title:

Data Engineer - Databricks in Healthcare

Locations:

Coimbatore / Bangalore / Hyderabad

Work Type:

Remote, with potential for Hybrid engagement

Work Timing:

3:00 pm – 11:00 pm IST (2–3 hours overlap with US West Coast)We are seeking experienced

Data Engineers

skilled in

Databricks

,

PySpark

, and

Python

to join a healthcare data-focused team. The ideal candidates will design and maintain scalable data pipelines and solutions for complex healthcare datasets, leveraging

Azure Cloud

and advanced distributed data processing frameworks. Strong client-facing communication is essential, as you will interact directly with US-based stakeholders.

Key Responsibilities

• Design, develop, and maintain large-scale data processing systems using Databricks and PySpark.
• Build and optimize robust, scalable data pipelines for data ingestion, cleaning, transformation, and storage from diverse sources.
• Collaborate with business and technical stakeholders to analyze requirements and translate them into technical solutions.
• Troubleshoot and enhance the performance of distributed data pipelines.
• Ensure all data engineering solutions adhere to healthcare data governance, privacy (e.g., HIPAA), and security standards.
• Operate effectively in an offshore setup, supporting some overlap with US West Coast working hours.
• Deliver clear technical documentation and participate in agile team processes.
  

Must-Have Skills

• Hands-on experience with Databricks for production data pipeline development.
• Proficiency in Apache Spark/PySpark and distributed data processing.
• Advanced Python scripting for data engineering tasks.
• Experience with Azure Data Lake, Azure Storage, or related cloud data services.
• Healthcare data knowledge, including compliance requirements and data standards.
• Strong client-facing verbal and written communication skills, with prior client interaction.
• Bachelor’s degree in Computer Science, Engineering, Data Science, or equivalent experience.
  

Preferred Qualifications

• Experience with large healthcare datasets (claims, EMR, HL7, FHIR, etc.).
• Exposure to CI/CD pipelines for data engineering workflows.
• Familiarity with Delta Lake, ML integrations in Databricks.
• Experience with Power BI or similar reporting tools.
• Relevant certifications (e.g., Azure Data Engineer Associate).
  

Salary Range:

₹8 LPA – ₹11 LPA

Experience Required:

4 – 5 YearsSkills: databricks,azure data lake,hl7,apache spark,python,azure storage,ci/cd pipelines,power bi,fhir,delta lake,healthcare data knowledge,emr,pyspark,ml integrations