Latest / Logosofia no Dia a Dia
GPU Scaling: The "Go Wide or Go Tall" Dilemma
In this episode, Herman and Corn dive deep into the engineering trade-offs of serverless GPU workloads. Using a real-world text-to-speech example on the Modal platform, they explore whether it’s better to scale horizontally with many small workers or vertically with a single high-end GPU like the H100. They break down the hidden costs of cold starts, the importance of memory bandwidth over raw compute, and how to find the "sweet spot" on the cost-efficiency curve to get the most bang for your buck. Episode 346 — myweirdprompts.com/n/346 (permanent short link: myweirdprompts.com/135098)
The skinny
The skinny isn't ready yet — notes appear once the transcript is processed.