Senior Generative AI Engineer (Databricks,Data Lake)

Title: Senior Generative AI Engineer (Databricks,Data Lake)

Location: 100% Remote

Job Type: Fulltime

Key Responsibilities

• Design and implement GenAI models (LLMs, multimodal, embeddings, and fine-tuning) for enterprise use cases.

• Architect and optimize data pipelines and workflows in Databricks for large-scale AI training and inference.

• Manage and maintain data lakes on Databricks, ensuring data quality, governance, and scalability for AI workloads.

• Collaborate with data scientists, ML engineers, and product teams to translate business problems into AI-driven solutions.

• Implement MLOps pipelines for model deployment, monitoring, and lifecycle management in Databricks.

• Explore and evaluate state-of-the-art GenAI techniques and integrate them into scalable architectures.

• Optimize performance, scalability, and cost-effectiveness of AI systems in cloud environments (AWS/Azure/GCP).

• Contribute to technical strategy, mentorship, and best practices in AI and data engineering.

Required Qualifications

• 8+ years of professional experience in AI/ML engineering, data engineering, or related roles.

• Strong expertise with Databricks platform, including:

o Databricks Workflows & Delta Lake

o Databricks ML Runtime & MLflow

o Databricks SQL

• Hands-on experience designing and managing data lakes at enterprise scale.

• Proficiency in Python (PySpark, Pandas, ML/AI libraries) and SQL.

• Solid background in Machine Learning, Deep Learning, and Generative AI frameworks (e.g., Hugging Face, LangChain, OpenAI APIs, TensorFlow, PyTorch).

• Experience with MLOps practices, CI/CD pipelines, and cloud-native deployments.

• Strong knowledge of cloud platforms (AWS, Azure, or GCP) and data services.

• Excellent problem-solving skills and ability to design scalable AI solutions for complex datasets.

Preferred Qualifications

• Databricks certifications (e.g., Databricks Certified Data Engineer Professional, Machine Learning Professional).

• Experience with vector databases (Pinecone, Weaviate, FAISS, Chroma).

• Experience with fine-tuning LLMs and integrating RAG (Retrieval-Augmented Generation) pipelines.

• Background in data governance, lineage, and compliance in large-scale AI systems.

• Contributions to open-source AI/ML projects or publications in the AI community.

Thanks

Vijay

Vijay.kumar@ampstek.com