Data Engineer - Databricks

Key Responsibilities

• Design and develop

  ETL/ELT pipelines

  for both

  batch and real-time data processing

  .
• Build and optimize

  data models

  ,

  data lakes

  , and

  data warehouse architectures

  to support

  analytics

  and

  machine learning

  workloads.
• Ensure

  data quality

  ,

  governance

  , and

  security

  across all workflows and environments.
• Collaborate with

  data scientists, analysts, and solution architects

  to enable

  data-driven decision-making

  .
• Integrate data from diverse sources, including

  databases, APIs, and streaming platforms

  .

• Monitor and optimize

  data pipelines for

  scalability

  ,

  reliability

  , and

  performance efficiency

  .
• Contribute to

  best practices

  , technical documentation, and the development of

  reusable components

  for data engineering.

Basic Qualifications

• 5+ years of experience

  in

  Data Engineering

  or related technical domains.
• Strong knowledge of

  SQL

  and proficiency in at least one programming language

  Python

  ,

  Scala

  , or

  Java

  .
• Hands-on experience with

  cloud data services

  such as

  AWS

  ,

  Azure

  , or

  Google Cloud Platform (GCP)

  .
• Experience with

  data pipeline orchestration tools

  like

  Apache Airflow

  ,

  Azure Data Factory (ADF)

  , or

  AWS Step Functions

  .
• Familiarity with

  big data frameworks

  such as

  Apache Spark

  or

  Apache Flink

  .
• Solid understanding of

  relational databases (SQL)

  and

  NoSQL databases

  .

Preferred Qualifications

• Experience with

  real-time data streaming technologies

  —

  Kafka

  ,

  Azure Event Hubs

  , or

  Google Pub/Sub

  .
• Exposure to

  Lakehouse architectures

  and tools like

  Databricks

  ,

  Snowflake

  , or

  BigQuery

  .
• Knowledge of

  data governance and cataloging

  tools such as

  Azure Purview

  ,

  AWS Glue

  , or

  Unity Catalog

  .
• Familiarity with

  machine learning pipelines

  and collaboration with

  data science teams

  .
• Strong

  problem-solving skills

  and ability to thrive in

  fast-paced, cloud-native environments

  .