John Schulman (OpenAI Cofounder) — Reasoning, RLHF, & plan for 2027 AGI

John Schulman, OpenAI co-founder and post-training lead, argues AGI could arrive in 2-3 years and outlines what careful deployment would actually require.

• Schulman estimates AI could replace his own job in ~5 years and acknowledges AGI in 2-3 years is a planning-worthy scenario.
• If AGI arrives unexpectedly soon, OpenAI’s plan is to pause further training and limit deployment scale until alignment is better understood.
• Coordination among the small number of frontier-model labs is feasible given capital intensity; unilateral safety pauses without coordination create race dynamics.
• Just ~30 labeled examples of a new capability boundary generalized across many untrained capabilities in early ChatGPT post-training.
• Fine-tuning on English-only data automatically produces reasonable behavior in other languages — cross-lingual generalization emerges without explicit training.
• Raters may be copy-pasting outputs from competing chatbots to complete labeling tasks, causing unintentional style convergence across providers.
• Post-training verbosity is partly a training artifact: single-turn preference labels favor completeness over concision, skewing models toward over-explaining.
• OpenAI’s model spec defines four stakeholder classes — end user, developer, platform (OpenAI), and humanity — with model itself potentially added in future.

2024-05-15 · Watch on YouTube