The Wonders of AI: We Are Retiring Our Bug Bounty Program

· ai · Source ↗

TLDR

  • Turso is ending its $1,000 data-corruption bug bounty because AI-generated slop PRs overwhelmed maintainers, making the cost of review unsustainable.

Key Takeaways

  • The program ran nearly a year; only 5 individuals were paid, including contributors who extended Turso’s Deterministic Simulator or used formal methods to find 10+ SQLite bugs.
  • Turso requires simulator extensions to demonstrate bugs, not just point them out – this kept quality high pre-AI flood.
  • Slop PRs range from manually injecting garbage bytes into DB headers to “discovering” that a SQL database executes SQL statements.
  • Slopmakers spend ~1 minute per submission; maintainers spend hours per review – with semi-infinite generation rate, no ratio works.
  • A vouching/auto-close system was tried but bots immediately opened issues demanding manual review, often resubmitting identical PRs from new accounts.

Hacker News Comment Review

  • Commenters broadly agree the real bottleneck is review bandwidth, not generation capacity – one framed this as proof that reading and understanding code is the scarce resource now.
  • A nominal submission fee (refunded on valid bug) was proposed as an alternative, but others noted even one wrongly rejected paid submission would trigger major PR backlash against Turso.
  • Several commenters pushed responsibility to GitHub/GitLab for bot account detection rather than individual maintainers, citing Hacktoberfest spam as a precedent.

Notable Comments

  • @arian_: “we automated finding bugs… now we’re automating rejecting submissions. at no point did anyone automate fixing the bugs.”
  • @Lalabadie: Points to a live bot honeypot repo and leaderboard tracking AI bounty-hunter agents in the wild.

Original | Discuss on HN