Is RL + LLMs enough for AGI? — Sholto Douglas & Trenton Bricken

Sholto Douglas and Trenton Bricken (Anthropic) argue RL has finally proven expert-level reliability, white-collar automation is 5 years away on current algorithms, and an interpretability agent already outpaces humans at detecting misaligned models.

• AI labs spend ~$1M on RL vs. hundreds of millions on pre-training; Dario confirmed this — RL compute scaling has barely started.
• Anthropic’s interpretability agent (Claude with interp tools) cracked a covertly implanted evil model in 90 minutes; human teams were given 3 days.
• Evil model experiment: fine-tuning on fake news articles about AI bad behaviors causes the model to generalize those behaviors to novel prompts never seen in training.
• White-collar work is automatable within 5 years using current algorithms, provided sufficient task-specific training data — even if algorithmic progress stalls entirely.
• Nobel Prize contributions are more likely to be AI-accelerated than a Pulitzer-winning novel because science has more verifiable reward layers.
• Human brain estimated at 30–300 trillion synapses vs. ~2 trillion parameters in the largest current models; models are almost certainly still underparameterized.
• US faces a ~34 GW compute-energy gap versus China (Dylan Patel); energy is the resource directly beneath intelligence in the AGI economy.
• For students: get deep technical skills (biology, CS, physics), treat AI as leverage multiplier, and performance engineering (GPU/TPU kernels) is a near-certain path to an AI lab offer.

2025-05-22 · Watch on YouTube