Senior Data Scientist

We are seeking a highly skilled and experienced Computer Scientist to join our dynamic team, focusing on the development, optimization, and application of Large Language Models (LLMs). The ideal candidate will have a proven track record of working on LLM projects, with extensive experience in implementing Retrieval Augmented Generation (RAG) models, utilising the LangChain framework, and optimizing LLM performance for various applications.

Key Responsibilities:

·      Design and implement Retrieval Augmented Generation (RAG) models to enhance the performance and accuracy of LLMs.

·      Utilize the LangChain framework for building efficient LLM workflows.

·      Develop methods for chunking and embedding non-structured and structured databases, storing these in vector databases such as Pinecone.

·      Conduct vector similarity searches to improve information retrieval processes.

·      Employ the llmware framework for citations and source verification, ensuring the accuracy of LLM responses by comparing them with source data.

·      Generate knowledge by prompting separate LLMs to produce information, aiming to improve overall LLM performance.

·      Possess in-depth knowledge of all main proprietary LLMs, including GPT-3, GPT-3.5Turbo, GPT-4, GPT-4Turbo, Claude 1, Claude 2, Llama, etc.

·      Apply advanced prompt engineering techniques such as Chain of Thought, Zero-shot CoT, Self Consistency, Least-to-most, AutoPrompt, Prefix Tuning, Prompt Tuning, and P Tuning to refine LLM outputs.

·      Ensure security measures, including encryption in transit and at rest, are upheld.

·      Optimize LLM performance through caching, efficient API calls, and cloud infrastructure optimization.

Mandatory Skills