Why a webhook can arrive more than once

The receipt for the receipt went missing

A payment provider sends “invoice settled” to a shop. The shop records the event and grants access. Its HTTP acknowledgment disappears on the way back. From the provider’s perspective, the delivery may have failed, so it retries.

If the shop treats each HTTP request as a new purchase, one payment can grant two entitlements, ship twice, or send duplicate refund requests. The duplicate request does not imply a duplicate payment. It can simply be uncertainty about whether the earlier notification arrived.

This is why webhook handling needs idempotency: repeated attempts at the same logical operation should not multiply its intended effect. Network delivery and business identity are separate questions.

Provider details vary. Stripe’s webhook documentation explicitly describes duplicates and ordering limits. BTCPay provides its own event payloads and validation examples. Do not transfer field names or signature algorithms between providers just because both send HTTP POST requests.

Authenticate before trusting the payload

A public endpoint can receive arbitrary requests. Verify the provider’s signature using the exact bytes and process required by its documentation. Then check the expected account or store context and the order/invoice mapping. Use provider reconciliation where necessary.

Logging an event ID is useful, but an in-memory list vanishes on restart and can race under concurrent delivery. The durable application needs uniqueness enforced at the storage boundary.

Two identities prevent two different duplicates

The event identity detects a replay of the same provider event. The business operation identity prevents multiple events about one invoice from creating multiple entitlements for one order.

A conceptual database transaction might look like this:

verify provider signature and expected store
read/reconcile current invoice state
begin transaction
  insert inbox(provider, store, event_id) with unique key
  if duplicate: finish successfully without new work
  lock the mapped order
  confirm expected amount, currency and eligible payment state
  insert fulfillment_intent(order_id, action) with unique key
  record the payment observation
commit
acknowledge the webhook

This is pseudocode. The insert and related state updates must have well-defined rollback behavior. A transient processing failure should not be permanently remembered as successful handling.

The fulfillment intent forms an outbox-like boundary. A worker or subsequent process performs the external effect and records completion. If that external call times out after succeeding, it still needs an idempotency key or reconciliation mechanism. A local database transaction cannot make a remote shipping or email service exactly-once by itself.

Out-of-order events need another guard: an older “processing” notification should not blindly overwrite a reconciled settled state. Consult current provider state and define allowed transitions instead of sorting solely by arrival time.

Simulate two deliveries and one effect

Use fictional records:

event E1: invoice I7 settled; maps to order O42
retry E1: identical event delivered again
new event E2: another settlement notification for I7

Expected result: E1 creates one durable fulfillment intent for O42. The retry finds the event uniqueness constraint and creates none. E2 has a new event identity, but the existing business-operation key prevents a second entitlement.

Now simulate a crash just before the database commit. Expected result: no partial successful receipt remains; a retry can complete the transaction. Simulate a crash after commit but before the HTTP acknowledgment. Expected result: the retry finds durable records and does not duplicate the effect.

These exercises expose the boundary that “just return 200” misses. A successful response should correspond to a safely recorded handling decision, with recovery available for unfinished external work.