Idempotency Is Easy Until the Second Request Is Different

· databases · Source ↗

TLDR

  • A replay cache handles clean retries; the real idempotency problem is concurrent requests, partial failures, downstream uncertainty, and same-key/different-body collisions.

Key Takeaways

  • Scope idempotency keys by tenant and operation name, not globally; a broken client’s abc-123 should never collide with another tenant’s.
  • Store a request_hash of the normalized, validated command so same-key/different-body conflicts are detectable and rejectable with 409 Conflict.
  • Use an atomic insert-on-conflict to acquire ownership before executing; read-then-write without a unique constraint allows two instances to both run the side effect.
  • Track statuses IN_PROGRESS, COMPLETED, FAILED_RETRYABLE, FAILED_REPLAYABLE, and UNKNOWN_REQUIRES_RECOVERY explicitly; each requires different retry behavior.
  • Hash the validated command DTO, not raw bytes: normalize field order, defaults, and API version; changing the hash algorithm breaks old retries silently.

Hacker News Comment Review

  • Commenters split on conflict policy: one camp argues any duplicate key should immediately return 409 and make it a client problem, avoiding server-side heroics entirely; the other says the key is a contract and the server must replay the stored response regardless of intent.
  • A real production incident was reported where a cart-derived idempotency key was reused across two different order amounts, causing silent data corruption in checkout – the exact same-key/different-body failure mode the article centers on.
  • Commenters noted that “adding an idempotency key” does not make an operation idempotent; it only deduplicates identical messages, and operations with differing economics under the same key are structurally not idempotent.

Notable Comments

  • @amluto: Recommends including a client-specified field on created records so clients can query status later, decoupling reconciliation from the idempotency key itself.

Original | Discuss on HN