Testing AI Models as Magic Boxes: Why Checks Fail - Michael Bolton

From Software Testing Unleashed - QA, DevEx & Quality Engineering · 28:07

Show by Richard Seidl | Software Development & Testing Expert

Testing AI models means testing a magic box: nobody has the source code, and the behavior is neither controlled nor understood. Michael Bolton calls that strongly magical, and with him I talk about what follows from it. His principles are simple: if something matters very much, don't depend on magic, and if you use it anyway, responsibility demands that you test it and maximize testability. Simple output checks don't stand a chance against a language model, so testing becomes experiencing, exploring and experimenting, like peeling an onion. And Penn and Teller's clear plastic cups show why we fall for these models in the first place: the insight we see in their text comes from us.

More episodes of Software Testing Unleashed - QA, DevEx & Quality Engineering

More episodes

PodSearch is a free, independent podcast search engine. Read about PodSearch, the privacy policy or contact us.