Catching Duplicate Submissions Before They Burn Credits

Logosofia no Dia a Dia · Sep 18, 2026 · 25m

When a webhook fires but the response never arrives, the form looks like it failed — so you resubmit, and now the pipeline is running the same job twice. This episode walks through the engineering fix: SHA-256 checksums for audio files, Unicode normalization before hashing text, and Jaccard similarity over character shingles for the fuzzy cases where a character changed between attempts. The conversation also draws a sharp line between transport idempotency (an event key that survives retries) and content dedupe (a fingerprint of what was actually submitted), explaining why you need both…