Stateless HTTP + loadbalancer + database architecture breaks for agentic LLM workloads; the fix is durable execution paired with addressable pub/sub channels.
Key Takeaways
Agents violate three stateless-web assumptions: long-running work, stateful context accumulation, and bi-directional interaction.
Durable execution tools (Temporal, Inngest, Restate) handle resilience but still require polling a database for client updates, reintroducing latency and load.
WebSockets solve routing once but lose the address on reconnect; pub/sub channels are durable addresses both client and server connect to by name.
LLMs amplify the pain: responses are non-deterministic and token-expensive, making retries and database-threaded streaming unacceptable workarounds.
Proposed stack: durable execution + pub/sub channels + stateless HTTP, each handling a distinct layer.
Hacker News Comment Review
Discussion is thin with no substantive architectural debate; one commenter flags the article leans heavily on the word “durable” (13 uses), suggesting the framing may be more rhetorical than rigorous.
A skeptical commenter argues the core insight is obvious: treating LLMs like deterministic computation causes problems, and “shitty but uncannily useful search” may be the more practical mental model for most LLM apps.
Notable Comments
@throwaway27448: frames real LLM utility as “shitty but uncannily useful search,” questioning whether agentic architecture is the right lens at all.
