Introducing asynchronous processing
Moving side effects out of the critical response path — resilient, traceable and idempotent.
What is this? · Playbook
A repeatable approach to a recurring challenge — situation, steps, decision points and validation. How we do it, not why. Go to overview
When this playbook applies.
Operations trigger side effects that are slow or unreliable: notifications, exports, calls to third-party systems. Kept synchronous, they tie response time and success to the slowest system involved.
Objectives
- Response time is decoupled from slow side effects.
- Side effects are resilient to failures and retries.
- Operations remain traceable.
Typical risks
- Duplicate effects caused by at-least-once delivery.
- Lost jobs without retries or proof of delivery (fire-and-forget).
- Queues and workers that are invisible in production.
- Eventual consistency that the user interface doesn't account for.
Before we build.
- Separate the critical response path from side effects that can be offloaded.
- Define delivery semantics and an idempotency strategy.
- Plan monitoring for the queue, workers and failed jobs.
How we proceed.
Only outside the critical path
Whatever is critical for the response stays synchronous. Only side effects that can be decoupled move into the queue.
Idempotency is mandatory
Every job is idempotent: an idempotency key is checked and persisted in the same transaction as the effect. That way, at-least-once delivery doesn't produce duplicate effects.
Visibility from the start
Throughput, run times and failed jobs are visible. Retries and dead-letter handling are explicit.
Handle eventual consistency deliberately
The user interface communicates that an effect is in progress (for example with 202 Accepted) instead of pretending to be immediately consistent.
Questions that need an answer.
Is the operation really outside the critical response path?
Is the job idempotent?
Are retry and failure paths defined?
Does the user interface account for eventual consistency?
Validation
- Duplicate delivery demonstrably produces no duplicate effect.
- Failed jobs are visible and can be retried.
- The latency of the critical path is decoupled and measured.
Common mistakes
- Making critical logic asynchronous and losing consistency.
- Fire-and-forget without retries or proof of delivery.
- Forgetting idempotency — duplicate bookings, duplicate emails.
- Running queues and workers without monitoring.
When we deliberately take a different approach.
- If an operation is small, fast and critical for the response, it stays synchronous — asynchrony would be unnecessary complexity.
- For very simple, infrequent tasks, a scheduled batch can be cheaper than a queue.
Continue your engineering journey.
Related concepts, decisions, playbooks and perspectives — as one connected path, not a list of links.
Engineering decisions
Perspectives
Facing a similar challenge?
Playbooks show how we think. For your specific project, talk to our management — technical, no sales pitch.
