The Other Half of Hugging Face: Why BERT Still Out-Downloads Llama

My Weird Prompts · Sep 21, 2026 · 22m

Everyone names Llama, Gemma, Mistral, and DeepSeek — but BERT is the second-most-downloaded model on Hugging Face, and encoder-only models collectively out-download decoder-only models nearly three to one. This episode digs into the taxonomy problem hiding in plain sight: two different animals sharing one foundational task, text in and text out. We trace the arc from BERT's pre-train-then-fine-tune breakthrough through ModernBERT, NeoBERT, and mmBERT, look at the Ettin benchmark results where a 150M encoder beats a 400M decoder at classification, and tour the crowded world of ASR cleanup…