Senior Data Engineer

Note:

About the Company

We are an AI and Data Consulting Startup transforming how businesses leverage technology through four core service lines:

• Consulting Services:

  AI Strategy, Automation, and Digital Transformation for enterprises.

• SaaS Platform Development:

  Building a business application suite similar to Odoo and Zoho that is AI-native and user friendly.

• Data Lakehouse Solutions:

  Unified data pipelines for aggregation, cleaning, governance, and advanced analytics.

• Government Contracting:

  Developing secure, compliant AI solutions for the public sector.

Our tech stack includes Python, TypeScript, React, Next.js, Go, Rust, Azure, Kubernetes, Spark, MLflow, Postgres, graph databases, and vector stores.

We're a small, fast-moving team delivering enterprise-grade solutions with startup agility.

Role Overview

You will design, build, and optimize data lakehouse solutions, enabling analytics, AI, and MLOps across multiple clients.

Key Responsibilities

• Design lakehouse architectures (Bronze/Silver/Gold) using Azure Databricks, Delta Lake, or Iceberg
• Build scalable PySpark pipelines for batch and streaming data
• Implement data quality, validation, and lineage tracking
• Integrate pipelines with MLflow for experiment tracking
• Optimize Spark performance and cloud cost
• Ensure data governance, PII protection, and security compliance
• Mentor developers and document patterns and runbooks

Required Skills

• Apache Spark (PySpark)

  and

  SQL

  (complex joins, CTEs, optimization)

• Delta Lake / Iceberg

  and

  Azure Data Stack

  (ADLS, Synapse, Databricks)

• MLflow

  integration for MLOps pipelines

• Kafka / Event Hubs

  for streaming data
• Strong

  Python

  and

  data modeling

  (dimensional, Data Vault)

• 5+ years

  hands-on in data engineering with production-grade systems

Nice to Have

• Vector databases, dbt, Airflow, Terraform,

  FinOps

  exposure, and

  Great Expectations

  data validation
• Experience with

  data governance

  technologies (such as MS Purview or similar)
• Certifications:

  Azure Data Engineer / Databricks DE

Education

• UG:

  B.Tech / B.E. – Computer Science / Information Technology

• PG:

  M.Tech / M.E. / Any Postgraduate (preferred)

Why Join Us

• Impact:

  You'll have significant influence over our data architecture and enable AI/ML capabilities across the organization.

• Growth:

  Help shape and build our data-driven culture from the ground up.

• Learning:

  Work with the latest in AI, code, and data technologies.

• Culture:

  Flat hierarchy, collaborative, and outcome-driven team.