Fine-Tuning at 4-Bit vs 16-Bit: What It Really Costs
What actually happens when you fine-tune a quantized model instead of the original weights? This episode walks through the mechanics of QLoRA versus full-parameter fine-tuning, the VRAM multipliers that decide whether a model fits on one consumer GPU or a rack of H100s, and the real cost tables for 8B and 70B models in 2026. We cover where quantization errors come from, how much of the quality loss quantization-aware initialization can recover, why 3-bit is the awkward middle child, and the safety-alignment degradation that never makes it into the cost comparison. Plus: the frameworks people…
The skinny
The skinny isn't ready yet — notes appear once the transcript is processed.