Designing Idempotent Event Consumers

Message brokers usually promise at-least-once delivery. That is a useful guarantee, but it means the same event can reach a consumer more than once. A payment, email, or inventory update must therefore be safe to repeat.

This guide focuses on the decisions that survive contact with production: clear boundaries, observable behavior, and a feedback loop that reveals when an assumption is wrong.

The problem worth solving

Duplicates appear after timeouts, consumer restarts, visibility-window expiry, and producer retries. Checking an in-memory set is not enough because the process can crash between the business write and the acknowledgement.

The useful move is to make the hidden constraint explicit. Write down what must stay correct, what can be delayed, and how the system should behave when a dependency fails. That turns a vague idea into something a team can test.

A practical implementation

Store an event identifier in the same transaction as the business change. A unique constraint turns the second delivery into a harmless no-op. Acknowledge the message only after the transaction commits.

async function handle(event: OrderPaid, db: Database) {
  await db.transaction(async (tx) => {
    const inserted = await tx.processedEvents.insertOnce(event.id)
    if (!inserted) return
    await tx.orders.markPaid(event.orderId, event.paidAt)
  })
}

The example is intentionally small. In a real project, add structured logs, metrics around the failure path, and tests for retries or partial results. Keep the interface narrow so the implementation can change without forcing every caller to change too.

What to measure

Measure the outcome rather than activity. For software, that may be latency, error rate, queue depth, or recovery time. For product work, it may be activation, retention, or the number of useful conversations. Review the signal on a regular cadence and record what changed.

Takeaway

Assume delivery can repeat. Put the deduplication record beside the state change, and let the database enforce the invariant.

Start with the smallest version that can teach you something, make its behavior visible, and improve it from evidence. That rhythm is more dependable than trying to design the final answer in one pass.

Further reading

• AWS Builders Library: Avoiding overload in distributed systems
• Apache Kafka design documentation

Explore more Systems articles from this journal.