Latest / Logosofia no Dia a Dia
MoE vs. Dense: The VRAM Nightmare
The AI world is obsessed with Mixture of Experts models, but dense transformers are quietly staging a comeback. This episode breaks down the brutal tradeoffs: MoE wins on training compute but loses on VRAM, fine-tuning stability, and edge deployment. We explore why the "free lunch" of massive parameter counts comes with a hidden tax, and where each architecture actually makes sense for developers. Episode 2067 — myweirdprompts.com/n/2067 (permanent short link: myweirdprompts.com/340143)
The skinny
The skinny isn't ready yet — notes appear once the transcript is processed.