GenAI Evaluation: One Model, Three Goals - Deepshikha, Anamika Mukhopadhyay
From Software Testing Unleashed - QA, DevEx & Quality Engineering · 27:58
Show by Richard Seidl | Software Development & Testing Expert
GenAI evaluation turns into a question of better or worse once every answer a model gives could be correct, and that is where Deepshikha and Anamika Mukhopadhyay begin. We talk about why one model needs three different definitions of good as a customer bot, a clinical summarizer and a coding agent, and why most teams they meet in workshops measure whatever their tools give them by default. A faithfulness score like 0.87279 left a room of CXOs asking whether it was a rounding problem; a 12 percent drop in issue resolution and 200 extra tickets did not. And a prepared Reddit post shows how an indirect prompt injection slips past an agent that had blocked every direct one.
More episodes of Software Testing Unleashed - QA, DevEx & Quality Engineering
More episodes
- Contract-Driven Testing in Banking: Realistic Mocks - Phil Royston
- Building Trust in Teams: Start Small - Michael Kutz, Linda Van De Vooren
- Mental Skills: Testers Can't Outsource Their Brain to AI - Ard Kramer
- Testing AI Models as Magic Boxes: Why Checks Fail - Michael Bolton
- Dark Side of Testing: The Failures We Cause Ourselves - Bart Knaack
- Quality Hunting: Testing for Information Instead of Bugs - Rik Marselis
- On-Site Testing Conferences: Why Networking Needs Presence - Okan Çakmak
- You Need a Test Automation Strategy Before AI - Michaël Pilaeten
- Agentic Systems in Testing: Why Humans Stay in the Loop - Nishan Portoyan
- Welcome to Software Testing Unleashed
PodSearch is a free, independent podcast search engine. Read about PodSearch, the privacy policy or contact us.