Job
Description
Job Description AI Architect with at least 5 years of experience designing and deploying enterprise-grade AI/ML solutions , specializing in Databricks, MosaicML, and Large Language Models (LLMs ). Expertise in building scalable AI architectures that integrate deep learning, natural language processing (NLP), and MLOps best practices to enable real-time decision-making and predictive insights. Proven ability to collaborate with cross-functional teams to operationalize AI models while ensuring data security, scalability, and performance in cloud environments. Key Qualifications AI/ML Architecture & Deployment: Extensive experience in designing and deploying end-to-end AI pipelines using Databricks and MosaicML. Expertise in model training, fine-tuning, and inferencing of LLMs, ensuring optimal performance and cost-efficiency in production environments. Databricks Expertise: Strong hands-on experience with Databricks for data processing, model training, and model serving. Proficient in using Databricks Workflows, MLflow, Delta Lake, and advanced SQL/Scala/PySpark for building scalable AI pipelines. MosaicML Proficiency: Proven expertise in utilizing MosaicML for fine-tuning and training foundation models, leveraging model compression and distributed training techniques to reduce latency and enhance model performance. LLM & NLP Expertise: In-depth knowledge of Large Language Models (LLMs), including OpenAI, GPT models, BERT, LLaMA, and custom fine-tuning techniques. Experienced in building conversational AI, semantic search, text classification, and other NLP-based applications. Model Optimization & MLOps: Experience in implementing MLOps pipelines with tools like MLflow, Kubeflow, and Airflow to automate model training, versioning, deployment, and monitoring. Skilled in optimizing model inference with techniques such as quantization, pruning, and distributed inferencing. Data Pipeline & Feature Engineering: Expert in designing feature stores and scalable data pipelines using Apache Spark, Databricks Delta, and AWS Glue to facilitate efficient feature generation, model retraining, and data versioning. Cloud AI/ML Infrastructure: Proficiency in AWS, Azure, and GCP with expertise in setting up AI environments using EC2, S3, EKS, Lambda, and SageMaker. Ability to design fault-tolerant and highly available AI/ML infrastructure on cloud platforms. AI Governance & Security: Experience in implementing AI governance frameworks, ensuring compliance with AI ethics, bias mitigation, and explainability. Knowledge of data privacy, GDPR, and model interpretability in enterprise environments. Preferred Experience Experience with RAG (Retrieval Augmented Generation) pipelines and semantic search Familiarity with vector databases such as Pinecone, FAISS, and Weaviate Exposure to LangChain, Hugging Face, and OpenAI APIs Experience in building custom AI/LLM solutions tailored to enterprise use cases Technical Skills AI/ML Platforms: Databricks, MosaicML, SageMaker, Azure ML, GCP AI Programming & Frameworks: Python, PyTorch, TensorFlow, Hugging Face, PySpark, SQL MLOps & Orchestration: MLflow, Kubeflow, Airflow, Docker, Kubernetes, CI/CD LLM & NLP Models: GPT-3/4, BERT, LLaMA, Falcon, BLOOM, T5, LangChain Databricks Tools: Databricks SQL, Delta Lake, AutoML, Unity Catalog Cloud Platforms: AWS (S3, EC2, Lambda), Azure, GCP Version Control & Automation: Git, Jenkins, Terraform, CloudFormation Certifications (Preferred but not mandatory) Databricks Certified Professional Data Engineer/AI Engineer MosaicML Certification in Model Training & Optimization AWS Certified Machine Learning Specialty Microsoft Certified: Azure AI Engineer Associate
