Senior Data Engineer with Python & AI Level

Hiring: Senior Data Engineer Python & Agentic AI

Positions:

We are looking for experienced Senior Data Engineers with strong expertise in Python, PySpark, Databricks, Cloud Data Engineering and Agentic AI development. The ideal candidate should have hands-on experience designing and developing scalable data engineering solutions, building enterprise-grade data pipelines, and integrating Generative AI and autonomous agent-based workflows into data platforms.

The candidate should be capable of driving technical solutions, developing modern data architectures and leveraging AI-assisted coding tools to improve engineering productivity.

Key Responsibilities

• Design, develop and maintain scalable data pipelines using Python, PySpark and Apache Spark.
• Build and optimize ETL/ELT workflows using Databricks and cloud-based data engineering platforms.
• Design and implement data lakehouse, data warehouse and modern data architectures.
• Develop and optimize complex SQL queries, data transformations and data models.
• Build and deploy Agentic AI applications using frameworks such as LangGraph, LangChain, CrewAI, AutoGen or Semantic Kernel.
• Develop multi-agent workflows with tool calling, API integrations, planning, memory and orchestration capabilities.
• Integrate LLMs, RAG pipelines, vector databases and enterprise data sources into data engineering solutions.
• Leverage AI-assisted development tools such as GitHub Copilot, Cursor and Claude Code to accelerate coding, debugging and development.
• Implement data quality, governance, monitoring and performance optimization practices.
• Work with cross-functional teams and stakeholders to deliver scalable enterprise solutions.
• Follow engineering best practices using Git, CI/CD and collaborative development workflows.

Mandatory Technical Skills

• Strong programming experience in Python.
• Advanced SQL and query optimization.
• Hands-on experience with PySpark / Apache Spark.
• Strong experience in Databricks.
• Cloud Data Engineering experience in Azure / AWS / GCP.
• Practical experience in Agentic AI development and multi-agent workflows.
• Experience with ETL/ELT pipelines and data lakehouse architecture.
• Knowledge of data modelling and data warehousing.
• Experience using AI coding assistants such as GitHub Copilot, Cursor, Claude Code or equivalent.
• Git and collaborative software development practices.

Good to Have

• LangGraph, LangChain, CrewAI, AutoGen or Semantic Kernel.
• Generative AI, LLMs, RAG and prompt engineering.
• Vector databases such as Pinecone, FAISS, Azure AI Search or ChromaDB.
• Kafka, Hadoop and distributed data processing.
• Azure Data Factory, AWS Glue, GCP Dataflow or equivalent.
• Databricks Delta Lake, Unity Catalog and Medallion Architecture.
• Docker, Kubernetes, MLflow and CI/CD.
• Data governance, observability and data quality frameworks.

Experience & Leadership Expectations

Manager:

Senior Manager:

Candidate Profile

We are looking for professionals who have worked on real-time enterprise data engineering and Agentic AI implementations. Candidates should have practical development experience rather than only theoretical knowledge or certifications.