Data Engineer

Crossing Hurdles

Position:

Role Responsibilities:

• Design and maintain scalable ETL/ELT pipelines that ensure datasets are clean, reliable, and ML-ready.
• Validate, enrich, and transform datasets across diverse formats including JSON, CSV, and Parquet.
• Manage data contracts, schema evolution, and version control to maintain consistency across pipelines.
• Work with PostgreSQL/SQLite and distributed processing systems like Spark or DuckDB.
• Orchestrate workflows using tools such as Airflow or equivalent schedulers.
• Optimize data pipelines for performance and reliability using Python and pandas.
• Collaborate with researchers and engineers to align data infrastructure with analytics and AI research needs.
  

Requirements:

• Strong background in data engineering, computer science, or information systems.
• Proficient in Python, pandas, and SQL.
• Hands-on experience with relational databases such as PostgreSQL or SQLite.
• Working knowledge of distributed data processing (Spark or DuckDB).
• Experience with workflow orchestration tools like Airflow.
• Familiarity with common data formats (JSON, CSV, Parquet).
• Strong understanding of schema design, data contracts, and version control using Git.
• Passion for building reliable, reproducible, and scalable data pipelines that power analytics and ML systems.
  

Key Domains:

If you specialize in any of the following, you are encouraged to apply:

• ETL/ELT Pipelines:

  Data transformations, enrichment, validation

• Data Engineering:

  Workflow orchestration, distributed processing, schema management

• Database Systems:

  PostgreSQL, SQLite, query optimization

• AI & ML Data Support:

  ML-ready datasets, reproducible dataset creation

• Data Infrastructure:

  Airflow, Spark, DuckDB, Parquet pipelines

Application Process:

• Apply for the job role
• Await the official message/email from our recruitment team (typically within 1-2 days)