Data Scientist and LLM Operations Specialist - Contractual Role

Job Description: As a candidate for this role, you will be responsible for the following key responsibilities: - Scope, develop, expand, operate, and maintain scalable, reliable, and safe generative AI solutions. - Design and execute prompt engineering experiments to optimize Large Language Models (LLMs) for various use cases. - Collaborate with Subject Matter Experts (SMEs) to evaluate prompt effectiveness and align AI solutions with business needs. - Understand and apply offline and online evaluation metrics for LLMs, ensuring continuous model improvements. - Evaluate production models using live data in the absence of ground metrics, implementing robust monitoring systems. - Monitor LLM applications for model drift, hallucinations, and performance degradation. - Ensure smooth integration of LLMs into existing workflows, providing real-time insights and predictive analytics. For a Data Scientist, the qualifications required include: - Proven experience in data science, with expertise in managing structured and unstructured data. - Proficiency in statistical techniques, predictive analytics, and reporting results. - Experience in applied science in fields like Natural Language Processing (NLP), Machine Learning (ML), Deep Learning (DL), or Multimodal Analysis. - Strong background in software development, data modeling, or data engineering. - Deep understanding of building and scaling ML models, specifically LLMs. - Familiarity with open-source tools such as PyTorch, statistical analysis, and data visualization tools. - Experience with vector databases and graph databases is a plus. For an AI/ML Engineer, the qualifications required include: - Solid experience in business analytics, data science, software development, and data engineering. - Expertise in Python and frameworks such as PyTorch, TensorFlow, or ONNX. - Hands-on experience working with LLMs and Generative AI, including model development and inference patterns. - Proven ability to design scalable systems leveraging LLMs, particularly in distributed computing environments. Preferred Skills: - Experience in prompt engineering and prompt optimization. - Expertise in running experiments to evaluate generative AI performance. - Knowledge of production-level monitoring tools for ML models, including drift detection and mitigation strategies. - Excellent problem-solving skills and ability to work cross-functionally with data scientists, engineers, and SMEs. - Experience with safety, security, and responsible use of AI. - Experience with red-teaming (adversarial testing) of generative AI. - Experience with developing AI applications with sensitive data such as PHI, PII, and highly confidential data.,