Google’s IO 2024 keynote pushed Search further toward AI-generated answers via AI Overviews, reducing traffic to source websites.
Key Takeaways
AI Overviews already carry an estimated ~10% error rate, yet Google is expanding this answer-synthesis model across Search.
The shift moves Google from a traffic router to a self-contained answer engine, breaking the implicit crawl-for-traffic deal with publishers.
Website owners report AI summaries surfacing their content incorrectly while referral traffic declines.
The move mirrors Facebook’s mid-2010s attempt to make the web synonymous with its own platform, which ultimately failed.
Hacker News Comment Review
Core tension: sites allow Googlebot access in exchange for traffic; if Google eliminates that exchange, blocking crawlers becomes rational, potentially accelerating the breakdown of the web’s indexing ecosystem.
Commenters split on sympathy: some note the open web is already bloated with ads and subscribe modals, weakening the case for defending it; others see real harm to independent publishers and skilled developers who are being replaced by AI-generated summaries.
Decentralized search came up as an unbuilt alternative, with general agreement that no viable option exists and that dependency on a single corporate gatekeeper was a structural mistake made years ago.
Notable Comments
@DeusExMachina: frames the crawl-for-traffic relationship as symbiotic and questions whether Google has modeled the incentive collapse correctly.
@hartator: points out Google actively litigates against being scraped itself, via the Google v. SerpApi case, highlighting the double standard.
