Testing the Unpredictable: QA for Agentic AI

Logosofia no Dia a Dia · Jul 23, 2026 · 23m

Traditional software testing assumes determinism — run the same test, get the same result. Agentic AI shatters that assumption. This episode maps the emerging QA landscape for probabilistic systems: from golden datasets and LLM-as-judge to trajectory evaluation and adversarial prompting suites like Garak. We explore what carries over from traditional testing, what requires entirely new methodologies, and what a sane minimum testing stack looks like for teams shipping agentic systems. Episode 4444 — myweirdprompts.com/n/4444 (permanent short link: myweirdprompts.com/238681)