Latest / Logosofia no Dia a Dia
When a Token Isn't a Word Anymore
Everyone talks about tokens for context windows and pricing, but what does a token actually mean when it's not a piece of text? This episode breaks down how VQ-VAE compresses images into patch tokens, how neural audio codecs turn waveforms into discrete IDs at 50-100 tokens per second, and why code tokenization needs syntax-aware approaches. We explore why the old "one token = ¾ of a word" heuristic is dead for multimodal models, how Gemini and GPT-4V handle mixed token types in a single context window, and what developers should know about budgeting context across different modalities…
The skinny
The skinny isn't ready yet — notes appear once the transcript is processed.