Data Engineer

Job Summary

Data Engineer

Key Responsibilities

• Design, develop, and maintain

  ETL pipelines

  for structured and unstructured data.
• Work extensively with

  Databricks, PySpark, Python, and SQL

  for data integration, transformation, and processing.
• Build and optimize

  data models

  for reporting, analytics, and advanced use cases.
• Develop and manage

  data warehouse solutions

  leveraging

  PostgreSQL and AWS services

  (e.g., S3, Redshift, Glue, Lambda).
• Perform

  data exploration, profiling, and quality checks

  to identify gaps and anomalies.
• Collaborate with business and technical teams to gather, analyze, and document

  data requirements

  .
• Implement

  unit testing (UT cases)

  and validation frameworks to ensure data accuracy and reliability.
• Maintain

  technical documentation

  , including data flow diagrams, design specifications, and metadata.
• Support data governance and ensure compliance with organizational and security standards.
  
• Track and manage tasks, issues, and progress using

  JIRA

  or similar tools.

Required Skills & Qualifications

• Proven experience in

  ETL development

  and data engineering projects.
• Strong programming skills in

  Python

  and

  PySpark

  .
• Proficiency in

  SQL

  (including performance tuning and complex queries).
• Hands-on experience with

  Databricks

  and

  AWS Cloud ecosystem

  .
• Solid understanding of

  data warehousing concepts, data modelling techniques, and database design

  .
• Experience with

  PostgreSQL

  or similar relational databases.
• Strong knowledge of

  data exploration, profiling, and debugging large datasets

  .
• Familiarity with

  unit testing

  and test automation for data pipelines.
• Excellent skills in preparing

  technical documentation

  .
• Experience with

  Agile methodologies and tools like JIRA

  .
• Strong communication and collaboration skills for cross-functional teamwork.
  

Preferred Qualifications

• Experience with

  big data ecosystems

  (Spark, Hive, Kafka).
• Exposure to

  CI/CD pipelines for data workflows

  .
• Knowledge of

  data governance, security, and compliance practices

  .
• Familiarity with BI/Visualization tools (e.g., Power BI, Tableau).