Databricks AI Engineer

Key Responsibilities

• Design, develop, and deploy AI/ML and Generative AI solutions using the Databricks platform.
• Build and implement LLM-powered applications, including Retrieval-Augmented Generation (RAG) solutions.
• Develop scalable data processing and AI pipelines using PySpark, Python, and Spark SQL.
• Utilize Databricks Mosaic AI capabilities for developing and deploying AI applications.
• Implement Vector Search, embedding models, and semantic search capabilities.
• Integrate Large Language Models (LLMs) with enterprise data sources and applications.
• Develop and optimize prompt engineering strategies for LLM-based applications.
• Use MLflow for model tracking, experimentation, evaluation, and deployment.
• Implement MLOps practices for model deployment, monitoring, and lifecycle management.
• Work closely with clients and stakeholders to understand requirements and provide technical solutions.
• Participate in client discussions, technical design sessions, and solution demonstrations.
• Troubleshoot and optimize AI application performance, scalability, and reliability.

Mandatory Skills

• Strong hands-on experience with Databricks.
• Experience in Python programming and PySpark.
• Practical experience in AI/ML or Generative AI development.
• Hands-on experience with Large Language Models (LLMs).
• Experience in developing RAG-based applications.
• Knowledge of embeddings, vector databases, and semantic search.
• Experience with MLflow or equivalent model lifecycle management tools.
• Experience in developing and deploying AI solutions in cloud environments.
• Strong analytical, problem-solving, and communication skills.