Senior Gen AI Data Engineer

Kellton

6 - 10 years

15 - 27 Lacs

gurugram

Posted:4 days ago| Platform:

Apply

Skills Required

azure data factory gen ai data engineering etl python etl pipelines pyspark vector database retrieval augmented generation sql

Work Mode

Work from Office

Job Type

Full Time

Job Description

Key Responsibilities

Develop and maintain data ingestion pipelines using Azure Data Factory (ADF) and Databricks for structured and unstructured data.
Create notebooks to process PDF and Word documents, including extracting text, tables, charts, graphs, and images.
Apply NLP / Embedding Models (e.g., OpenAI, Hugging Face, sentence-transformers) to convert extracted content into embeddings.
Store embeddings and metadata into Vector Databases (e.g., FAISS, Pinecone, Milvus, Weaviate, ChromaDB).
Design and implement semantic search and retrieval workflows to enable prompt-based query capabilities.
Optimize ETL pipelines for scalability, reliability, and performance.
Collaborate with data scientists and solution architects to integrate GenAI capabilities into enterprise applications.
Follow best practices for code quality, modularity, and documentation.

Required Skills & Experience

Proven experience in Azure Data Factory (ADF) and Databricks for building ETL/ELT

workflows.

Strong programming experience in Python (pandas, PySpark, PyPDF, python-docx, OCR libraries, etc.).
Hands-on experience with Vector Databases and semantic search implementation.
Understanding of embedding models, LLM-based retrieval, and prompt engineering.
Familiarity with handling multi-modal data (text, tables, images, charts).
Strong knowledge of data modeling, indexing, and query optimization.
Experience with cloud platforms (Azure preferred).
Strong problem-solving, debugging, and communication skills.

Nice to Have

Experience with knowledge graphs or RAG (Retrieval-Augmented Generation) pipelines.
Exposure to MLOps practices and LLM fine-tuning.
Familiarity with enterprise-scale document management systems.

More Jobs at Kellton

Golang Developer

Gurugram

5 - 7 yrs

INR 9 - 18 Lacs

Senior Data Engineer - Snowflake, SQL, SSIS

Hyderabad, Telangana, India

8 - 8 yrs

Salary: Not disclosed

Senior Angular Developer

Gurugram

5 - 8 yrs

INR 10 - 17 Lacs

Quality Assurance Manager

Gurugram

10.0 - 14.0 yrs

INR 20 - 30 Lacs

Senior Qa Automation Test Engineer

Hyderabad, Gurugram

6.0 - 10.0 yrs

INR 8 - 18 Lacs

Mock Interview

Practice Video Interview with JobPe AI

Start Python Interview

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.