How to Catch an LLM's Bad Writing Habits

Logosofia no Dia a Dia · May 5, 2026 · 29m

Daniel asked a deceptively practical question: how do you systematically analyze a corpus of podcast transcripts to catch what a script-writing LLM overdoes — repeated words, stale jokes, dialogue patterns that need more variety? This episode walks through the full spectrum of techniques, from quick Python frequency counts with NLTK and spaCy to embedding-based clustering with sentence transformers and LLM-as-judge qualitative passes. Herman and Corn discuss when simple analysis is enough, when you need the heavy machinery, and — crucially — how to avoid optimizing for metrics that make…