Job
Description
As an experienced LLM Engineer, your role will involve leading the deployment and optimization of self-hosted Large Language Models (LLMs) such as LLaMA, Mistral, and Falcon on on-premise GPU servers. You will be responsible for deploying and fine-tuning LLMs for low-latency, high-efficiency inference, setting up GPU-accelerated servers for AI workloads, implementing model quantization, developing APIs for model inference, automating model deployment, fine-tuning and training models, monitoring system performance, and collaborating with cross-functional teams. Key Responsibilities: - Deploy and optimize self-hosted LLMs for low-latency, high-efficiency inference. - Set up GPU-accelerated servers for AI workloads using CUDA, TensorRT, and vLLM. - Implement model quantization (GPTQ, AWQ, bitsandbytes) for efficient memory usage. - Develop APIs for model inference using FastAPI, Flask, or Hugging Face TGI. - Automate model deployment with Docker, Kubernetes, and CI/CD pipelines. - Fine-tune and train models using LoRA, QLoRA, or full fine-tuning methods. - Monitor system performance and optimize for scalability and security. - Work with cross-functional teams to integrate LLMs into applications. Qualifications Required: - 3+ years of experience in AI/ML model deployment. - Strong knowledge of Python, PyTorch, and TensorFlow. - Hands-on experience with LLM inference frameworks (e.g., vLLM, Text Generation Inference). - Experience with NVIDIA GPU acceleration (CUDA, TensorRT, Triton). - Proficiency in Linux, Docker, and Kubernetes. - Knowledge of security best practices for AI deployments. - Experience with distributed computing and API development. In addition, if you have experience with on-premise AI infrastructure, familiarity with vector databases (FAISS, Chroma, Pinecone) for RAG applications, and contributions to open-source AI/ML projects, it would be considered as preferred qualifications for the role. As an experienced LLM Engineer, your role will involve leading the deployment and optimization of self-hosted Large Language Models (LLMs) such as LLaMA, Mistral, and Falcon on on-premise GPU servers. You will be responsible for deploying and fine-tuning LLMs for low-latency, high-efficiency inference, setting up GPU-accelerated servers for AI workloads, implementing model quantization, developing APIs for model inference, automating model deployment, fine-tuning and training models, monitoring system performance, and collaborating with cross-functional teams. Key Responsibilities: - Deploy and optimize self-hosted LLMs for low-latency, high-efficiency inference. - Set up GPU-accelerated servers for AI workloads using CUDA, TensorRT, and vLLM. - Implement model quantization (GPTQ, AWQ, bitsandbytes) for efficient memory usage. - Develop APIs for model inference using FastAPI, Flask, or Hugging Face TGI. - Automate model deployment with Docker, Kubernetes, and CI/CD pipelines. - Fine-tune and train models using LoRA, QLoRA, or full fine-tuning methods. - Monitor system performance and optimize for scalability and security. - Work with cross-functional teams to integrate LLMs into applications. Qualifications Required: - 3+ years of experience in AI/ML model deployment. - Strong knowledge of Python, PyTorch, and TensorFlow. - Hands-on experience with LLM inference frameworks (e.g., vLLM, Text Generation Inference). - Experience with NVIDIA GPU acceleration (CUDA, TensorRT, Triton). - Proficiency in Linux, Docker, and Kubernetes. - Knowledge of security best practices for AI deployments. - Experience with distributed computing and API development. In addition, if you have experience with on-premise AI infrastructure, familiarity with vector databases (FAISS, Chroma, Pinecone) for RAG applications, and contributions to open-source AI/ML projects, it would be considered as preferred qualifications for the role.
