Data Engineer

Job Title: Data Engineer

EST overlap (1pm IST-10:30pm IST)

Location: Remote

Position: Full-time

Department: Engineering

Position Overview

Data Engineer

You will be responsible for one or more software and data products, participating in architectural decisions, design, development, and evaluation of new technical solutions. This role also involves leading a small team of developers.

Key Responsibilities

• Manage and optimize data and compute environments to enable efficient use of data assets
• Design, build, test, and deploy scalable and reusable systems capable of handling large volumes of data
• Lead a small team of developers; conduct code reviews and mentor junior engineers
• Continuously learn and help teach emerging technologies to the team
• Collaborate with cross-functional teams to integrate data into broader applications

Required Skills

• Proven experience designing and managing data flows
• Expertise in designing systems and APIs for data integration
• 8+ years of hands-on experience with

  Linux, Bash, Python, and SQL

• 5+ years working with

  Spark

  and other components in the

  Hadoop

  ecosystem
• 5+ years of experience using

  AWS cloud services

  , particularly:

• EMR

• Glue

• Athena

• Redshift

• 4+ years of experience managing a development team
• Deep passion for technology and enthusiasm for solving complex, real-world problems using cutting-edge tools

Additional Skills (Preferred)

• BS, MS, or PhD in Computer Science, Engineering, or equivalent practical experience
• Strong experience with Python, C++, or other widely-used languages
• Experience working with petabyte-scale data infrastructure
• Solid understanding of:
• Data organization: partitioning, clustering, file sizes, file formats
• Data cataloging: Hive/Hive Metastore, AWS Glue, or similar
• Background working with relational databases
• Proficiency with Hadoop, Hive, Spark, or similar tools
• Exposure to modern data stack technologies like dbt, airbyte
• Experience building scalable data pipelines (e.g., Airflow is a plus)
• Strong background with AWS and/or Google Cloud Platform (GCP)
• Demonstrated ability to independently lead projects from concept through launch and ongoing support