SQLite loadable extension that adds durable queues, streams, and pub/sub with single-digit ms cross-process delivery by watching WAL file changes instead of polling a broker.
Key Takeaways
Enqueue and business write commit atomically in the same SQLite transaction; rollback drops both, eliminating the dual-write problem Redis+Celery introduces.
Push semantics come from stat(2)-polling the .db-wal sidecar: any WAL commit wakes listeners in ~1 ms with no daemon or wire protocol.
Requires journal_mode = WAL; honker_bootstrap() refuses non-WAL files because DELETE mode exclusive locks would serialize every enqueue behind active readers.
Feature set covers at-least-once work queues, durable streams with per-consumer offsets, ephemeral notify/listen, named locks, rate limiting, and a leader-elected cron scheduler.
Bindings ship for Python, Node.js, Bun, Ruby, Go, Rust, and Elixir; all wrap the same Rust-compiled loadable extension so schema compatibility is pinned by interop tests.
Hacker News Comment Review
Commenters probed the WAL checkpoint edge case: when SQLite truncates the WAL back to zero, the stat(2) signal could fire on a size decrease, raising a question about whether the poller filters shrinks or risks missing events.
The choice of stat(2) over kqueue/FSEvents/inotify drew scrutiny; the author’s README defense is that FSEvents silently drops same-process writes on macOS, making kernel notifications unreliable for the common publisher-and-listener-in-one-process case.
A recurring counter-argument is that same-machine processes could use OS IPC (Unix sockets, etc.) for the notification path and reserve SQLite only for durable storage, trading simplicity for lower latency on the hot path.
Notable Comments
@tuo-lei: raises WAL checkpoint truncation as a potential lost-event window when the file shrinks back to zero under stat polling.
@nzoschke: “most of these need a queue and scheduler… always pining for the elegance of the Postgres solutions” – validates the exact gap honker targets for multi-app SQLite shops.