Retry Logic Design for Idempotent and Non-Idempotent Agent Tasks

An agent that double-charges a customer $847 didn't fail because it retried too many times or waited too little between attempts. It failed because nobody classified the Stripe call as non-idempotent before the retry logic got written. That single missing step, sorting every tool call into "safe to retry" or "not safe to retry" before touching timeout values, is the decision that separates a production agent from a demo that quietly corrupts state the first time a network hiccups.

The damage pattern is almost always the same shape. A payment call times out. The retry logic fires again without checking whether the first charge actually landed. Two charges post, one order exists, and the finance team finds out three days later during reconciliation.

Agents make this worse than a normal REST API retry ever could. A retry in a typical web app reruns one endpoint. An agent retry can rerun a whole chain: by the time the timeout hits, the model may have already called three tools in sequence, sent an email, written a database row, and kicked off a webhook. Rerunning the sequence sends the email out twice, gives the database a duplicate row, and fires the webhook again downstream.

Then there's nondeterminism, which is absent from ordinary distributed systems retry logic. Large language models produce close to the right tool call almost all the time, but not always the exact same call. On a retry, the model can generate a new unique identifier, alter a payload field, or otherwise produce a call that looks equivalent to a human reading the logs but registers as a brand-new request to the downstream system. Developers have documented this as the "ghost order" problem: a tool designed to be idempotent on order ID still produces three separate orders because the agent generated a fresh ID each time instead of reusing the one from the failed attempt.

And the surface area for all this has grown fast. Tool calls showed up in 21.9% of agent traces in 2024, up from just 0.5% in 2023, a dramatic multi-fold jump in a single year. Average steps per task climbed too. More tool calls per task means more chances for a mid-chain failure, and more chances for a naive retry to make things worse instead of better.

What idempotency means at the tool-call layer

An operation is idempotent if running it once and running it five times leave the system in the same state. "Set the account balance to $500" is idempotent. "Add $500 to the account balance" is not, because running it twice adds a meaningful additional cost.

Some operations are naturally idempotent. GET requests are, because reading data doesn't change it. DELETE is usually idempotent too, since deleting something that's already gone just returns the same logical outcome as the first deletion. POST is the troublemaker. A POST that creates a record or charges a card produces a new effect every single time it runs, unless something is specifically built to stop that.

Idempotency is a property of the entire chain the call travels through, not just one call sitting in isolation. It's a property of the entire chain the call travels through. A documented case of a duplicate subscription charge shows this clearly: Stripe's own idempotency layer worked exactly as designed, and the customer still got charged twice, because the retry was being orchestrated one level above where the idempotency key actually lived. The key protected the API call. Nothing protected the workflow that decided to make the call again.

The instinct to fix this by editing the prompt doesn't hold up either. Telling the agent "only call the payment tool once" sounds reasonable until you realize the agent genuinely does not know the first call succeeded. It's a missing piece of external state: the tool has no way to say "I already did this," so the model has nothing to reason against. It's a missing piece of external state. The tool has no way to say "I already did this," so the model has nothing to reason against.

Classifying every tool call before writing a single retry rule

Before any retry ceiling or backoff curve gets configured, every tool needs a tier assigned to it, based on what happens if a retry lands on top of a call that already succeeded.

Controlled writes are the low-risk end: CRM field updates, internal task creation, draft generation. These can retry safely as long as there's an idempotency strategy and a hard ceiling on attempts.

High-impact writes need more care: payments, refunds, order submissions, access changes, contract actions. These should require reconciliation before any retry fires, and often a human approval gate before the system tries again.

Irreversible actions get the strictest rule of all: deleting regulated records, sending a legally binding notice. These should never retry automatically after an uncertain outcome, full stop.

This classification belongs in the tool's schema or metadata, decided at design time, not buried in a prompt or left to the model's judgment mid-run. A schema can be reviewed by a human auditor. An in-context reasoning chain, generated fresh every time the model runs, cannot be audited the same way, and that difference matters enormously once real money or real legal exposure is on the line.

A retryable flag on every tool error response is one more thing worth building early. A simple retryable: true or retryable: false on the returned error stops the agent from hammering a permanently broken call over and over, wasting time and money chasing a failure that was never going to resolve on its own.

And watch for the nondeterminism wrinkle even inside a well-classified tool. An order management system idempotent on order ID is still non-idempotent in practice if the agent's instructions say "generate an order ID" instead of "reuse the order ID from the failed attempt." Three retries, three ghost orders, and a tool that was designed correctly the whole time.

The idempotency key pattern: how to make a non-idempotent tool call safe to retry

The fix for a non-idempotent call is to give it a key that stays the same across every retry attempt. It's to give it a key that stays the same across every retry attempt, generated from durable, deterministic state rather than a fresh timestamp or randomly generated identifier. Something like {workflowRunId}:{stepId}:{actionType} works, because it can be reproduced exactly the same way even if the whole process restarts from scratch.

Making this work takes cooperation across three layers, not just one. The agent runtime has to generate that key and hold onto it reliably, including across a crash and resume. The tool execution layer has to check a deduplication store before doing anything: if the key already exists and the prior call succeeded, hand back the cached result without re-executing anything; if the key exists and the prior call failed permanently, hand back that same error instead of trying again. And the tool interface itself needs to accept the key and either pass it straight through to whatever API it's calling, or enforce it as a unique constraint on its own database writes.

Plenty of external APIs already support this natively. Stripe takes an Idempotency-Key header directly. PayPal uses a PayPal-Request-Id header for the same purpose. Square accepts the key as a field in the request body. Pass the deterministic key through to any of these, and a duplicate call returns the original response instead of processing a second time.

Internal actions that don't come with this built in, an email send, an internal database write, need the equivalent built by hand. A dedup table keyed on the idempotency key as its primary key does the job. Order matters here: claim the slot in that table before running the action, not after. If the process crashes between executing the action and logging that it happened, claiming first means the next retry sees the claimed slot and skips re-execution, instead of quietly duplicating the action because nothing was recorded yet.

Saga patterns for multi-step workflows where a middle step fails

Idempotency keys solve the "did this exact call already happen" problem. They don't solve what happens when a workflow succeeds at step two and then fails permanently at step three.

Take a standard order flow: reserve inventory, charge the customer, send a confirmation. Each of those three steps can be made individually idempotent. But if the charge succeeds and the confirmation email fails for good, the customer has been charged with no confirmation in hand. Retrying the whole sequence from the top might now fail at the inventory step, because stock already dropped to zero on the first pass.

A saga handles this by pairing every step with a compensating action, an explicit undo, so the system reaches consistency through deliberate reversal instead of a database transaction spanning services that were never designed to support one.

For the order example, the pairing looks like this: reserving inventory compensates with releasing the reservation, charging payment compensates with issuing a refund, sending confirmation compensates with sending a cancellation notice. If confirmation fails permanently, the saga executor walks backward through what already succeeded, refunding the payment and releasing the inventory hold, so the customer never sits in a charged-but-unconfirmed limbo waiting for someone in support to notice.

Backoff, jitter, retry budgets, and dead letter queues: the mechanics that bound retry behavior

None of the mechanics below matter until transient and permanent errors get sorted first. Retrying a permanent failure, one already flagged retryable: false, burns time and compute while delaying the moment a human sees the failure and could fix it.

For the transient failures worth retrying, exponential backoff with jitter is the standard approach, where each retry waits longer than the last, say 2 seconds, then 5, then 12, and a small random offset gets added to each wait so that hundreds of workflows hitting the same downstream outage don't all retry in the same instant and hammer the recovering service right back into the ground. That synchronized pile-up is usually called the thundering herd, and jitter exists specifically to prevent it.

Retry limits need two dimensions, not one: a maximum attempt count, and an overall retry budget measured in time. A customer-facing agent might get a 30-second deadline for the whole interaction. A background reconciliation job might get several minutes. Bound both, or one of two bad things happens: a time-sensitive customer flow gets starved waiting on retries that should've given up already, or a background task retries forever with no ceiling in sight.

When retries finally exhaust, the dead letter queue entry needs to carry the original input, checkpoint data from whatever steps did complete, the specific step that failed, full error detail, and the retry history leading up to the failure, so a human can act on it. Done right, a reviewer can resubmit the workflow from the exact point it broke, instead of re-running the entire pipeline from zero and risking every idempotency and saga problem already covered above.

What production agents look like once these patterns are in place

A large-scale study by Pan et al., surveying 306 practitioners and running 20 in-depth case studies across 26 domains, gives a grounded picture of what production agents actually look like today. Sixty-eight percent of production agents execute at most 10 steps before needing a human to step in. Seventy-four percent depend primarily on human evaluation to judge whether the work was done right. Reliability, not capability, tops the list of development challenges practitioners report.

With proper verification loops, retry logic, and graceful degradation in place, overall task completion rates in the 90 to 97% range have been cited as achievable. Getting to 99.9%, the reliability bar familiar from traditional infrastructure uptime targets, isn't realistic for agent systems as they exist now, and treating that as an achievable near-term target sets teams up to chase the wrong number.

The best-performing agents run with the smallest, most narrowly scoped tool sets available. They escalate to a human frequently in early iterations, and that escalation data is what calibrates the confidence thresholds that eventually let autonomy expand safely. Teams that skip the escalation phase and hand an agent full autonomy from day one lose the feedback loop that would have told them where the model actually breaks.

Monitoring has to expand along with it. Uptime, CPU, and memory tell you the infrastructure is healthy. They say nothing about whether the agent is doing its job. That requires tracking task completion rate, token consumption, tool call success rate, context utilization efficiency, and the quality of the reasoning trace itself, metrics that don't exist in a traditional ops dashboard because they didn't need to, until agents started making decisions instead of just serving requests.

Deploying agents with these guarantees built in: what the runtime and hosting layer must provide

None of the patterns above survive contact with production if the hosting layer isn't built to carry them. A saga executor needs somewhere durable to persist which compensating actions have and haven't run. An idempotency key needs to survive a restart. The runtime holding it can't lose state every time a process recycles. Retry budgets need a clock that keeps counting across a crash, not one that resets to zero the moment the container restarts.

The unit of isolation matters just as much as the persistence. An agent-powered business scales correctly when each user gets an isolated, persistent agent environment of their own, rather than every user's workflows sharing a single execution context where one user's retry storm or one user's stuck saga can spill over and affect somebody else's order. Get that isolation wrong, and every pattern described above, the classification tiers, the idempotency keys, the compensating actions, the backoff curves, still works correctly in isolation and still fails in aggregate, because the boundary meant to contain the damage wasn't actually there.