Database Engineer

Overview

Data Engineer

Key Responsibilities

• Design, develop, and maintain

  ETL/ELT pipelines

  .
• Build and optimize

  data models

  , data lakes, and data warehouses.
• Develop scalable solutions for collecting, processing, and storing large datasets.
  
• Work with structured and unstructured data from various sources.
  
• Ensure data quality, consistency, and reliability across systems.
  
• Collaborate with data analysts, data scientists, and software teams.
  
• Automate data workflows and improve data pipeline performance.
  
• Implement best practices for

  data governance, security, and compliance

  .
• Monitor pipeline performance and troubleshoot issues.
  

Required Skills

• Strong programming knowledge in

  Python

  ,

  Java

  , or

  Scala

  .
• Experience with

  SQL

  and

  NoSQL

  databases (e.g., PostgreSQL, MySQL, MongoDB).
• Hands-on experience with

  ETL tools

  and data pipeline frameworks.
• Experience with

  big data technologies

  (Hadoop, Spark, Hive, Kafka).
• Knowledge of

  cloud platforms

  such as:

• AWS

  (Glue, Redshift, S3, EMR)

• Azure

  (Data Factory, Synapse, Databricks)

• GCP

  (BigQuery, Dataflow, Dataproc)
• Familiarity with

  data warehousing concepts

  and

  dimensional modeling

  .
• Experience with version control (Git).
  
• Strong understanding of data structures, algorithms, and system design.
  

Preferred Qualifications (Optional)

• Experience with

  Airflow

  ,

  Luigi

  , or similar orchestration tools.
• Experience with

  Docker

  ,

  Kubernetes

  , and DevOps practices.
• Knowledge of

  Machine Learning pipelines

  (e.g., MLflow).
• Understanding of data security practices and GDPR.
  
• Experience with real-time processing using Kafka or Spark Streaming