Payment webhooks are inherently asynchronous and untrusted. When Stripe processes a subscription payment or creator escrow release, their infrastructure promises at-least-once delivery. If your server takes more than 5,000ms to respond due to a slow database query, or if an intermediate load balancer drops the connection, Stripe retries the exact same event multiple times over several hours.
On financial platforms like Curved Lab, an unhandled duplicate webhook means crediting a creator's wallet twice or sending redundant platform notifications. Here is how we engineered an atomic, distributed idempotency layer that guarantees every financial event is processed exactly once.
The Fallacy of In-Memory Deduplication
Many systems attempt to guard against duplicate webhooks by storing processed event IDs in an in-memory Set or local cache. In real production setups running across multiple Node.js worker processes or serverless instances, this approach fails immediately. Two concurrent retries from Stripe hit different application processes simultaneously, both find the cache empty, and both credit the user's account.
A resilient idempotency architecture requires distributed locking combined with persistent, atomic state transitions in the primary database.
