Data Engineer

Role Overview

automating and streamlining data processing pipelines

Key Responsibilities

• Own

  end-to-end data pipeline development

  , including data modeling, testing, scalability, operability, and performance metrics.
• Design, build, and maintain

  high-quality ETL/data pipelines

  that deliver business value.
• Ensure software quality by

  reviewing peer code

  , enforcing best practices, and maintaining coding standards.
• Define and promote

  best practices

  for product development, engineering processes, and coding standards.
• Collaborate closely with

  QA engineers

  to implement test automation and ensure reliable data systems.
• Deliver features in

  short development iterations

  following Agile methodologies.
• Continuously evaluate and introduce

  new tools, frameworks, and practices

  to improve data and code quality.
• Research, evaluate, and integrate

  third-party tools and libraries

  , and build internal tools to improve workflows.
• Participate actively in Agile ceremonies and contribute to continuous integration and delivery pipelines.
  

Required Experience & Skills

• 5 to 7 years of experience

  developing data pipelines and ETL processes.
• Strong understanding of

  database design and data engineering principles

  .
• Hands-on experience with

  automated test frameworks

  .
• Proficiency in the following technologies:
  

• Python

• Airflow, Luigi

  , or similar workflow orchestration platforms

• Relational databases

  such as PostgreSQL, MySQL, or equivalent

• Cloud platforms

  : AWS, Azure, or similar

• Version control systems

  : GitHub or equivalent

• Automated build and CI/CD tools

• Strong experience with

  Agile development practices

  , including Jira or Pivotal Tracker.
• Proven experience with

  continuous integration, automated testing, and test-driven development (TDD)

  .
• Ability to collaborate effectively in cross-functional teams and mentor junior engineers.