Local LLMs are how nerds now justify a big computer they don't need
TLDR
- Local LLMs are technically impressive but practically inferior to frontier rented models; most developers don’t need high-VRAM hardware for them.
Key Takeaways
- Local models like downscaled DeepSeek and gpt-oss-20b run on consumer hardware but lag significantly behind frontier models available via API.
- DHH argues evaluating a new computer by local LLM performance is spurious because all local models currently underperform for real dev work.
- Once developers test local models, they return to rented frontier models for the bulk of their work.
- RAM prices are rising due to AI demand; DHH says most developers, especially on Linux, need far less RAM than they assume.
- Small models are improving, but the claim is they are not yet reliable enough for daily developer workflows.
Why It Matters
- Developers risk overspending on 128GB VRAM machines for local inference that doesn’t outperform cheap API calls to frontier models.
- Rising RAM costs driven by AI infrastructure demand make the cost of overbuying hardware a real near-term financial consideration.
- The practical gap between local and frontier models means hardware purchase decisions justified by local LLM capability are likely to disappoint.
DHH · 2025-11-25 · Read the original
