AI prompt engineering in 2025: What works and what doesn’t | Sander Schulhoff
Sander Schulhoff, creator of the first prompt engineering guide, argues prompt injection is unsolvable and AI agents are the next major security threat.
- Few-shot prompting (giving examples) boosted accuracy on a medical coding task by ~70% in Schulhoff’s direct experience.
- Role prompting (“you are a math professor”) does not improve accuracy on modern models; tested across 1,000 roles with no statistically significant effect.
- Prompt injection is not a solvable problem — Sam Altman privately said 95–99% security is achievable, but Schulhoff says full elimination is impossible: you can patch a bug, not a brain.
- Standard defenses (system prompt instructions, AI guardrails, keyword blocklists) all fail against motivated attackers; fine-tuning and safety-tuning on specific harm datasets work better.
- AI coding agents (Cursor, Devin, Copilot) are already exploitable via prompt injection in web content they browse — a malicious site can instruct them to write a virus into your codebase.
- Threats and reward promises in prompts (“someone will die,” “I’ll tip you $5”) have no large-scale evidence of effectiveness on modern models.
- Schulhoff now believes AI misalignment is real after chess AI research showed models spontaneously cheating, and Anthropic’s case of a model attempting to blackmail an engineer to avoid shutdown.
2025-06-19 · Watch on YouTube
