5 Promptflow Jobs

As a ML Specialist GenAI at Claidroid Technologies Pvt. Ltd., your primary responsibility will be to design, develop, and deploy scalable Generative AI systems. This role involves implementing LLM agents, RAG architectures, and MLOps pipelines, and managing the full GenAI product lifecycle. You will lead AI developers, build solutions using LLMs, Diffusion Models, and GenAI agents, and fine-tune models with LangChain, LangGraph, and promptflow. Deployment will utilize Azure, containerization, and CI/CD practices. It will be your duty to ensure model observability, optimize performance and cost, apply ethical AI principles, and effectively communicate insights to stakeholders. This position is based in Pune/Trivandrum with options for WFH/Hybrid, working from 11 AM to 8 PM IST, suitable for candidates with under a 30-day notice period. To excel in this role, you should possess 5-10 years of experience in ML Engineering and Generative AI. Proficiency in technologies like LLMs, RAG, Prompt Engineering, LangChain, LangGraph, promptflow, and Autogen is essential. Strong skills in Python, PyTorch/TensorFlow, FastAPI, and AsyncIO are required. Experience with CI/CD pipelines (specifically GitHub Actions), Docker, Kubernetes, Azure cloud services (preferred), or AWS/GCP is advantageous. Additionally, expertise in MLOps, model deployment, monitoring, performance tuning, and understanding of Responsible AI principles, bias mitigation, and model explainability are crucial for this role. Claidroid Technologies is a leader in digital transformation, specializing in Enterprise Service Management and Enterprise Security Management solutions. With headquarters in India and offices in Helsinki, Finland, and the USA, we aim to deliver bespoke services to a wider audience globally. Joining our team offers competitive compensation, a hybrid working model, generous benefits such as comprehensive health insurance and performance bonuses, as well as ample opportunities for career growth in a dynamic and innovative environment. Be part of our collaborative and inclusive work culture that values your contributions and supports your professional development.,

Job Title: AI Engineer Quality & Performance Job Description: We are seeking a highly skilled AI Engineer - Quality & Performance to ensure the accuracy, robustness, and ethical integrity of AI-generated responses. In this role, you will leverage AI validation frameworks, adversarial testing tools, and responsible AI principles to assess and enhance AI systems. If you are passionate about AI security, performance evaluation, and responsible AI practices, wed love to hear from you! Key Responsibilities: AI Response Validation: Evaluate AI-generated responses for accuracy, coherence, relevance, and alignment with defined standards. AI Red Teaming: Utilize tools such as Pyrit, NVIDIA Garak, Giskard AI, Attest, and TruLens to conduct adversarial testing and identify vulnerabilities, biases, or security risks in AI models. AI Ground Truth Check: Develop and refine validation datasets using frameworks like PromptFlow, Cerberus, CheckList, and FairBench to ensure factual integrity and reliability of AI outputs. AI Performance Evaluation: Implement benchmarking strategies using tools such as OpenAI Eval, DeepEval, Dynabench, and LM Harness to measure AI model efficiency, robustness, and responsiveness. AI Quality Assurance: Apply automated validation frameworks such as DeepChecks, Great Expectations, and Robustness Gym to maintain high standards in AI response generation. Responsible AI Validation: Leverage tools like Fairlearn, Aequitas, and AI Fairness 360 to assess AI fairness, mitigate biases, and enforce ethical AI principles. Prompt Rewriting & Validation: Analyze and refine AI prompts using PromptFoo and Helium to optimize response quality and consistency across diverse applications. Qualifications: Education: Bachelors or Masters degree in Computer Science, AI, Engineering, or a related field (or equivalent practical experience). Experience: 2-8 years of experience in AI engineering, software development, or automation testing. Minimum 1 year of experience working with red teaming frameworks (e.g., Pyrite, Attest) and AI validation tools (e.g., Fairlearn, DeepChecks). Strong understanding of AI security, fairness principles, and adversarial testing methodologies. Technical Skills: Proficiency in programming languages such as Python, JavaScript, or relevant AI evaluation scripting. Experience in AI bias detection, robustness testing, and prompt engineering. Familiarity with machine learning workflows, AI policy frameworks, and ethical AI implementation. Key Competencies: Strong analytical and problem-solving skills in AI validation and red teaming. Attention to detail when assessing AI-generated responses. Creativity and critical thinking in designing and refining AI prompts. Excellent written and verbal communication skills for reporting AI evaluation results. Ability to work in a fast-paced, collaborative environment with multidisciplinary teams. Why Join Us? Work with cutting-edge AI validation frameworks to shape the future of responsible AI. Engage in AI security, fairness assessments, and red teaming to enhance AI resilience. Be part of an innovative and dynamic team at the forefront of AI model evaluation. Competitive compensation package with strong career growth opportunities.

Position Overview We are seeking an experienced AI Architect to lead the discovery phase for a complex AI agent platform serving a lot of users in the financial services sector. This role requires deep technical expertise in modern AI frameworks, strategic thinking for long-term architecture planning, and the ability to work directly with demanding clients to translate business requirements into actionable technical roadmaps. Key Responsibilities Discovery Phase Leadership Client Engagement : Work directly with the client to understand and document all use cases (that are required to be built) spanning semantic search, document processing, predictive modeling, and agentic analytics Requirements Analysis : Translate complex business needs into detailed technical specifications with accuracy requirements (including 100% accuracy for financial compliance use cases) Architecture Strategy : Design future-proof, modular architecture that prevents vendor lock-in while maximizing strategic flexibility Technical Architecture Design Hybrid AI Stack : Design and validate integration of DSPy + LangGraph + PromptFlow + Azure AI services Scalability Planning : Architect solutions for 100K user base with cost-effective licensing models Integration Strategy : Plan seamless integration with existing product ecosystem Technology Evaluation: Conduct comparative analysis of AI frameworks, providing evidence-based recommendations Deliverable Creation Technical Feasibility Studies : Comprehensive analysis for all the use-cases of the requirement Prototype Development : Build working demos demonstrating key capabilities and optimization approaches Cost-Benefit Analysis : Justify investment into a tech stacks by comparing it against other stacks for the long-term roadmap. Implementation Roadmap : Detailed phased approach from pilot to full production deployment Strategic Planning Long-term Vision: Create long term technology evolution plan preventing costly refactoring Risk Assessment : Identify and mitigate stack lock-in risks and technical dependencies Go-to-Market Strategy : Define pilot features for rapid market entry while building toward comprehensive platform Required Technical Expertise AI/ML Frameworks DSPy : Deep understanding of automated prompt optimization, few-shot learning, and algorithmic tuning LangGraph : Experience with multi-agent orchestration and complex workflow design Azure AI & PromptFlow : Proficiency in Microsoft's AI services and visual workflow tools RAG Architectures : Advanced knowledge of retrieval-augmented generation systems Cloud & Infrastructure Azure Ecosystem : Comprehensive understanding of AI Foundry, Cognitive Services, and enterprise scaling Microservices Architecture : Design of modular, swappable components API Design : RESTful services and integration patterns Performance Optimization : Large-scale system optimization and monitoring Financial Services Domain [Good to have] Regulatory Compliance : Understanding of financial data accuracy requirements and audit trails Document Processing : Experience with legal document parsing (LPAs, fund documents) Predictive Analytics : Investment modeling and risk assessment systems CRM Integration : Customer relationship management and sentiment analysis Required Experience Professional Background 8+ years in AI/ML architecture roles with enterprise clients Hands-on experience with modern AI frameworks (DSPy, LangGraph, or similar) Proven track record of leading discovery and implementation for complex AI implementations Client Management Executive Communication : Ability to present technical concepts to C-level stakeholders Requirements Gathering : Expert in translating business needs to technical specifications Stakeholder Management : Experience managing demanding, detail-oriented clients Documentation : Exceptional technical writing and presentation skills Technical Leadership Architecture Design : Led design of scalable AI systems serving 50K+ users Technology Evaluation : Experience conducting comparative analysis of AI platforms Prototype Development: Hands-on coding ability for proof-of-concept development Cost Estimation : Accurate project scoping and resource planning Preferred Qualifications Advanced Expertise PhD/MS in Computer Science, AI/ML, or related field Publications / Patents in AI optimization or enterprise AI architecture Speaking Experience at AI conferences or industry events Open Source Contributions to AI frameworks or libraries Industry Experience [Good to have] Private Equity/Investment Management domain knowledge Regulatory Technology experience with audit and compliance systems Enterprise AI Deployments at scale (100K+ users) Cost Optimization experience with AI workloads and licensing models Key Success Metrics Discovery Phase Outcomes Client Approval : Scott approves progression to development phase based on discovery results Technical Validation : All use cases of the requirement deemed technically feasible with proposed architecture Cost Justification : Clear ROI demonstration for 4x cost premium over SFDC alternative Timeline Adherence : Discovery completed within agreed timeframe and budget Architecture Quality Future-Proof Design : Architecture prevents vendor lock-in and supports long-term evolution Scalability Validation : 100K user performance and cost models validated Integration Feasibility : Seamless integration strategy with the product confirmed Accuracy Framework : 100% accuracy requirements for financial compliance addressed Application Requirements Portfolio Submission Architecture Samples : 2-3 examples of complex AI system designs you've led Case Studies: Detailed examples of discovery phase leadership with measurable outcomes Technical Writing : Samples of technical documentation for executive audiences Client References : References from previous discovery/consulting engagements Technical Assessment Architecture Design : Live design session for a sample use case from Scott's requirements Framework Knowledge : Deep-dive technical discussion on DSPy optimization approaches Business Acumen : Case study analysis of technology investment decisions Client Interaction : Mock discovery session with simulated challenging client requirements

Responsibilities: * Collaborate with dev team on API testing using GIT and CI/CD pipeline. * Develop automated tests with Python, PyTest, and frameworks.

Role & responsibilities We are seeking a highly skilled QA Automation Engineer with 3-5 years of experience in software development, SDET, or QA automation, with a strong focus on backend systems, APIs, or complex data pipelines. This role demands deep expertise in Python programming and a strong understanding of automation tools, CI/CD processes, and API testingparticularly within evolving AI-driven environments. Design, implement, and maintain robust automation frameworks using Python (Pytest or similar). Conduct thorough testing of backend systems , data pipelines, and RESTful APIs . Build test cases and scripts to support automated validation of AI-integrated services. Use tools like Postman or the requests library for API testing. Manage source control with Git and integrate tests into CI/CD pipelines (Jenkins, GitLab CI, GitHub Actions). Analyze and document defects with clarity, and collaborate with developers for resolution. Continuously adapt and learn new testing approaches for modern AI/ML-powered systems . Must-Have Skills 35 years of experience in software development, QA automation, or SDET roles. Strong hands-on programming skills in Python . Proven experience with automation frameworks like Pytest . Solid experience in REST API testing and tools like Postman or requests. Familiarity with CI/CD tools such as Jenkins, GitLab CI, or GitHub Actions. Excellent communication skills with the ability to articulate technical concepts clearly. Fast learner with the ability to adapt quickly to evolving AI/ML technologies . Preferred Qualifications Experience with cloud platforms (AWS, GCP, Azure), especially AI/ML services. Exposure to testing AI-based features or systems, especially in the telecom domain . Knowledge of UI automation tools like Selenium or Playwright (nice to have) Familiarity with structured testing of AI service pipelines and performance benchmarking. What We Offer Remote work flexibility Competitive compensation based on skill and experience Opportunity to work on cutting-edge AI-powered platforms A collaborative, learning-first culture with global project exposure