LLM - COBOL Developer 3+

What does day-to-day look like:

• Design, develop, and maintain efficient, high-quality COBOL code to train and optimize AI models.

• Conduct evaluations (Evals) to benchmark model performance and analyze results for continuous improvement.

• Evaluate and rank AI model responses to user queries across diverse domains, ensuring alignment with predefined criteria.

• Develop comprehensive explanations and rationales for evaluations, showcasing excellent reasoning and technical expertise.

• Lead efforts in Supervised Fine-Tuning (SFT), including creating and maintaining high-quality, task-specific datasets.

• Collaborate with researchers and annotators to execute Reinforcement Learning with Human Feedback (RLHF) and refine reward models.

• Design innovative evaluation strategies and processes to improve the model's alignment with user needs and ethical guidelines.

• Create and refine optimal responses to improve AI performance, emphasizing clarity, relevance, and technical accuracy.

• Conduct thorough peer reviews of code and documentation, providing constructive feedback and identifying areas for improvement.

• Collaborate with cross-functional teams to improve model performance and contribute to product enhancements.

• Continuously explore and integrate new tools, techniques, and methodologies to enhance AI training processes.

Requirements:

• Strong experience with COBOL programming language concepts.

• Industry experience and knowledge of code quality, formatting, and best practices of software development

• Experience with COBOL’s testing ecosystem, including unit, integration, and property-based testing.

• Familiarity with COBOL frameworks and libraries

• Knowledge of multi-threading and asynchronous programming in COBOL.

• Ability to work with architectural patterns and refactor code without introducing regressions.

• Strong debugging skills, including fixing memory and concurrency issues.

• 3+ years of overall work experience with 2+ years of relevant experience with COBOL

• Fluent in conversational and written English communication skills