Data Scientist and LLM Operations Specialist RAPSYS TECHNOLOGIES PTE LTD

5.0 - 9.0 years

0 Lacs

karnataka

On-site

You should have at least 5+ years of experience for this role. As a key member of the team, your responsibilities will include scoping, developing, expanding, operating, and maintaining scalable, reliable, and safe generative AI solutions. Your expertise will be crucial in designing and executing engineering experiments to optimize Large Language Models (LLMs) for various use cases. Collaborating with Subject Matter Experts (SMEs) to evaluate prompt effectiveness and align AI solutions with business needs will be an essential part of your role. You will need to understand and apply offline and online evaluation metrics for LLMs to ensure continuous model improvements. Additionally, you will be responsible for evaluating production models using live data, implementing robust monitoring systems in the absence of ground metrics, and monitoring LLM applications for model drift, hallucinations, and performance degradation. Your role will also involve ensuring the smooth integration of LLMs into existing workflows, providing real-time insights, and predictive analytics. For Data Scientists, we are looking for individuals with proven experience in data science, expertise in managing structured and unstructured data, proficiency in statistical techniques, predictive analytics, and reporting results. Experience in applied science in fields like Natural Language Processing (NLP), Machine Learning (ML), Deep Learning (DL), or Multimodal Analysis is preferred. A strong background in software development, data modeling, or data engineering is necessary, along with a deep understanding of building and scaling ML models, specifically LLMs. Familiarity with open-source tools such as PyTorch, statistical analysis, and data visualization tools is a plus, as well as experience with vector databases and graph databases. For AI/ML Engineers, we are seeking candidates with solid experience in business analytics, data science, software development, and data engineering. Expertise in Python and frameworks such as PyTorch, TensorFlow, or ONNX is required. Hands-on experience working with LLMs and Generative AI, including model development and inference patterns, is essential. You should have a proven ability to design scalable systems leveraging LLMs, particularly in distributed computing environments. Preferred skills for this role include experience in prompt engineering and prompt optimization, expertise in running experiments to evaluate generative AI performance, knowledge of production-level monitoring tools for ML models, including drift detection and mitigation strategies, excellent problem-solving skills, and the ability to work cross-functionally with data scientists, engineers, and SMEs. Experience with safety, security, and responsible use of AI, red-teaming (adversarial testing) of generative AI, and developing AI applications with sensitive data such as PHI, PII, and highly confidential data will be an advantage.,

Posted 1 week ago

Apply

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.