Senior Data Engineer

Role : Senior Data Engineer

Location : Hyderabad

Employment Type : Full-Time

The Senior Data Engineer will build and maintain the core data infrastructure for an enterprise AI platform. This role focuses on designing scalable data pipelines, developing knowledge graphs, and preparing structured and unstructured data for AI and LLM-based applications.

Roles & Responsibilities :

Data Pipeline Development :

Knowledge Graph Engineering :

- Design ontologies and graph schemas for complex enterprise relationships
- Implement entity resolution and relationship inference across data sources- Build APIs and query interfaces for graph traversal- Optimize graph storage and query performance for large-scale usage

Enterprise Data Integration :

- Extract and model enterprise metadata such as business rules and data dictionaries
- Parse and semantically index documents and code artifacts- Build integrations with enterprise APIs and internal platforms

AI & LLM Data Infrastructure :

- Prepare structured and contextual data for LLM consumption
- Design embedding strategies and manage vector databases for semantic search- Build memory and context management systems for stateful AI applications

Required Skills :

Core Requirements :

- 5+ years of Data Engineering experience with production-grade pipelines
- Strong Python skills (clean, testable, maintainable code) - MongoDB expertise (schema design, aggregation pipelines, indexing, performance tuning)- Vector databases experience (Qdrant, Pinecone, Weaviate, pgvector)- Document processing experience (chunking, metadata extraction, PDFs/Word/HTML; LangChain or similar)- Strong SQL Skills (complex Queries, Joins, Window Functions, Optimization)- ETL/ELT at scale (incremental loads, CDC, idempotent pipelines)- Pipeline orchestration tools (Airflow, Dagster, Prefect, or similar)

Good to Have / Strong Plus :

- Experience building production RAG pipelines
- Deep understanding of embedding models and dimensionality- Graph databases (Neo4j) and Cypher query expertise- LLM application development using LangChain or Lang Graph- Streaming systems (Kafka, Flink) for real-time pipelines

- Hybrid search (vector + keyword/metadata filtering)
- Apache Spark for large-scale transformations

Skills :

pipelines, rag, cdc, vector databases, metadata,flink,kafka,design,data Warehouse