Data Engineer Lead

Role:

Description:

Responsibilities:

• Design and maintain data architectures (data lakes, warehouses, real-time systems) using Databricks.
  
• Build and optimize ETL/ELT pipelines with Databricks & Apache Spark.
  
• Implement data governance, security, and compliance standards.
  
• Collaborate with data scientists, analysts, and stakeholders to deliver business-driven data solutions.
  
• Integrate Databricks with cloud platforms (AWS, Azure, GCP) and CRM/marketing systems (Salesforce, Veeva).
  
• Stay updated on Databricks, Delta Lake, and related technologies; drive process improvement.
  
• Lead, mentor, and train the data engineering team.
  
• Collaborate with analytics teams for KPI monitoring and reporting frameworks.
  

Required Skills & Experience:

• 12+ years in data engineering, with focus on Databricks, Apache Spark, and cloud platforms (AWS, Azure, GCP).
  
• Proficient in Python, Scala, SQL, ETL/ELT design, and pipeline orchestration (Airflow, ADF).
  
• Pharma domain knowledge, especially HCP/HCO entities.
  
• Expertise in data modeling (Medallion architecture), schema design, and database management.
  
• Experience in commercial pharma analytics preferred.
  
• Strong leadership, project management, and communication skills.
  
• Certifications in Databricks, AWS, Azure, or equivalent required.
  
• Bachelor’s in Computer Science, Data Science, or related; Master’s is a plus.