Data Engineer

Job Title: Data Engineer

Experience: 4+ Years

Location: Remote

Employment Type: Full-time

About the Role

Data Engineer

high-performance, auditable, and scalable

Key Responsibilities

• Design, build, and maintain

  Python-based ETL pipelines

  for large-scale data ingestion and transformation.
• Develop and optimize

  SQL models and transformations

  to support analytics, dashboards, and reporting workloads.
• Implement

  workflow orchestration

  using

  Prefect

  , integrating with

  Ray

  for distributed parallel task execution.
• Work with

  AWS services (S3, Aurora RDS, EKS)

  for data ingestion, orchestration, and storage.
• Containerize and deploy data pipelines using

  Kubernetes (EKS)

  and manage releases with

  Helm charts

  .
• Collaborate with cross-functional teams to ensure

  data accuracy, lineage, and auditability

  .
• Optimize pipeline performance, reliability, and cost efficiency using observability and monitoring tools.
• Maintain

  CI/CD pipelines

  using

  Bitbucket

  ,

  Terraform

  , and other infrastructure automation tools.
• Troubleshoot complex data and performance issues in distributed systems.

Required Skills

• Strong proficiency in

  Python (3.11)

  for data manipulation, scripting, and pipeline automation.
• Solid understanding of

  SQL

  and relational database design (PostgreSQL, Aurora RDS).
• Hands-on experience with

  Prefect

  for orchestration and

  Ray

  for distributed compute workflows.
• Experience in building and maintaining

  ETL/ELT pipelines

  and ensuring

  data quality, lineage, and validation

  .
• Familiarity with

  AWS cloud ecosystem

  — especially

  S3, RDS, Lambda, and EKS

  .
• Exposure to

  Docker/Kubernetes

  and cloud-native deployment practices.
• Experience with

  CI/CD automation

  using

  Bitbucket

  ,

  Helm

  , and

  Terraform

  .

Preferred Skills

• Experience with

  data modeling

  ,

  data warehousing

  , and

  analytical schema design (Star/Snowflake)

  .
• Understanding of

  distributed systems

  ,

  parallel computing

  , and

  scalable data architectures

  .
• Familiarity with

  observability and monitoring

  (Datadog, OpenTelemetry).
• Knowledge of

  data governance

  and

  security best practices

  for cloud environments.

Tech Stack

• Languages / Runtime:

  Python 3.11, SQL

• Containerization & Orchestration:

  Kubernetes (EKS)

• Workflow Orchestration:

  Prefect

• Parallel Task Runner:

  Ray

• Data Storage & Processing:

  AWS Aurora RDS, PostgreSQL (cached storage), S3 buckets

• CI/CD:

  Bitbucket using Helm chart and Terraform