Data Engineer

Key Responsibilities

• Design, develop, and optimize

  ETL/ELT pipelines

  .
• Manage

  batch and real-time (streaming) data ingestion

  using modern frameworks.
• Ensure

  data quality, integrity, and consistency

  across systems.
• Collaborate with data scientists and ML engineers to provide reliable datasets.
  
• Maintain

  data documentation, metadata, and lineage

  for transparency and governance.

Must-Have Skills

• Strong proficiency in

  SQL/NoSQL databases

  .
• Hands-on experience with

  Apache Spark

  and

  Apache Kafka

  .
• Expertise in

  Airflow

  (or similar orchestration tools).
• Solid experience in

  ETL/ELT pipeline development

  .

Good-to-Have Skills

• Exposure to

  vector databases

  (e.g., Milvus, Pinecone).
• Experience with

  data lineage and governance tools

  .
• Familiarity with cloud platforms (AWS, GCP, or Azure) for data engineering.
  

Experience Required

• 2 to 4 years

  of relevant experience in data engineering.