Data Engineer

About the Role:

Data Engineer with strong experience in Databricks

Key Responsibilities:

• Design, develop, and optimize scalable ETL/ELT data pipelines using

  Apache Spark on Databricks

• Collaborate with data scientists, analysts, and business stakeholders to gather data requirements and deliver clean, reliable datasets
• Build and manage data models, data lakes, and data warehouses (preferably in

  Delta Lake architecture

  )
• Implement data quality checks, monitoring, and alerting systems
• Optimize performance of data workflows in Databricks and ensure cost-efficient use of cloud resources
• Integrate with various structured and unstructured data sources (APIs, databases, files, etc.)
• Contribute to best practices and standards for data engineering and cloud development
  

Required Qualifications:

• Bachelor's or Masters degree in Computer Science, Engineering, or a related field
• 5+ years of experience as a Data Engineer or similar role
• Hands-on experience with

  Databricks

  and

  Apache Spark

• Strong programming skills in

  Python

  and/or

  Scala

• Proficient in

  SQL

  and data modeling (e.g., star/snowflake schemas)
• Experience with cloud platforms like

  Azure

  ,

  AWS

  , or

  GCP

  (Azure preferred if using Azure Databricks)
• Solid understanding of

  Delta Lake

  ,

  Lakehouse architecture

  , and

  data governance

Preferred Skills:

• Experience with

  CI/CD for data pipelines

• Knowledge of

  MLflow

  ,

  Unity Catalog

  , or

  Databricks Workflows

• Familiarity with orchestration tools like

  Airflow

  ,

  Azure Data Factory

  , or

  Prefect

• Exposure to BI tools like

  Power BI

  ,

  Tableau

  , or

  Looker

• Experience with real-time data streaming tools (e.g.,

  Kafka

  ,

  Event Hubs

  )

What We Offer:

• Competitive salary and performance-based bonuses
• Flexible working arrangements
• Opportunities to work with cutting-edge data technologies
• A collaborative and innovative work environment