Durable Execution Patterns for Long-Horizon Agent Tasks

Long-horizon agent tasks fail for a boring reason: the infrastructure running them was built for a completely different job. Receive a request, compute an answer, hand it back, forget everything ever happened. Agents break that contract on both ends. They carry state forward across dozens of steps, and they run for minutes, hours, sometimes days. Closing that gap is an engineering problem, and it has a name: durable execution. It's an engineering problem, and it has a name: durable execution.

The mismatch appears the moment something goes wrong. A standard web server assumes completion in milliseconds: read a request, do some work, write a response, release every resource. That model has no concept of "still running three hours from now" or "waiting on a human who might reply in three days." Agents live in exactly that territory. They pile up reasoning steps, tool outputs, and subagent results as they go, and none of it is disposable the way a single request-response cycle is.

Picture the worker process running a twenty-step agent task, and it dies halfway through. Every intermediate result, every tool call's output, every subagent's partial progress: gone. Restarting from step one doesn't just waste tokens and time. If any of those steps wrote to an external system, filed a support ticket, sent an email, committed a code change, restarting from scratch means doing those things again. A second ticket. A duplicate email. A commit nobody asked for. The failure compounds because nothing marked the line between "done" and "not yet done."

Pausing creates a separate version of the same problem. An agent waiting on human approval at step 7 of 20 can't just sit there holding a worker process hostage. The approval might take thirty seconds. It might take three days. Infrastructure built around fast request-response cycles has no graceful way to handle that: it either times out or burns compute doing nothing.

The scale of this is growing fast, and not slowly. METR's tracking of AI task-completion horizons found the length of task a model can finish at a 50% success rate has roughly doubled every seven months, moving from about 4 seconds in 2019 toward projections north of 16 hours by 2026. Agents are getting handed work that stretches far past anything stateless infrastructure was ever built to hold. A survey out of RUC-NLPIR sorts that difficulty into three tiers: tasks with many coupled steps inside one context window, tasks that outgrow a single window and stretch across hours or days, and open-ended streams of tasks with no fixed endpoint. Each tier exposes a different hole in the infrastructure underneath it.

SWE-Bench Pro puts a number on the ceiling this creates: current best models resolve about 40% of complex, multi-step coding tasks. That number is measuring how well the harness around the model holds up over the full length of a task, not the model itself. It's measuring how well the harness around the model holds up over the full length of a task, and for a lot of real work, it doesn't hold up well enough yet.

What "durable execution" provides for agents

Durable execution is a runtime model where progress gets checkpointed as it happens, control flow replays deterministically, failure-prone steps retry on their own, and wait states survive a process restart without losing anything. That's a different animal from wrapping a function call in a retry loop. A retry loop reruns the whole thing from scratch. Durable execution resumes from the last completed step and leaves everything that already succeeded alone, because re-running a step that already had a side effect, a card already charged, a message already sent, is exactly the failure mode the whole approach exists to avoid.

A durable agent runtime has to get four things right.

Reliability comes first, and it's non-negotiable. A crash, a deploy, a transient network blip anywhere in the agent loop should never erase work already done. Resumption picks up from the last completed step with all prior state intact.

Pausability matters just as much, maybe more, given how often agents wait on humans. An agent waiting for approval has to actually stop: freeing the worker, freeing its resources, and coming back exactly where it left off once the input arrives, whether that's five seconds or five days later.

Bounded memory keeps long sessions from choking on their own history. Context and tool outputs grow with every single step, and left unmanaged, that growth has no ceiling. History needs compression, and large outputs need to live somewhere other than the live context window.

Idempotent tool calls round it out. Every tool invocation has to survive being replayed. If a step reruns after a crash, it can't fire off a second version of whatever external effect it already caused the first time.

None of this happens by accident, and no single layer fixes it alone. The RUC-NLPIR survey frames long-horizon agency as a system-level capability, shaped by the harness around the model: loops, workflows, context and memory handling, tool design, orchestration, verification. A better model doesn't get you there by itself. Neither does a better harness. They have to move together, or the whole system stalls out under its own weight.

Azure's documentation on Durable Task draws a useful line between two kinds of agent workflows. Deterministic workflows, a scheduled report, a release checklist, follow a fixed, code-defined sequence, so they're predictable and fully replayable. Agent-directed workflows, open-ended research, exploratory debugging, hand control of branching to the model itself, which makes replay harder, since the model might make a different call on the second pass than it did on the first. Both benefit from checkpointing. They just need different assumptions baked in about what a "replay" is even allowed to look like.

Put together, the architecture that actually works is built around durable step boundaries, idempotent tool execution, explicit signals for when a step is genuinely done, and replayable histories. Not around hoping the process stays alive long enough to finish.

Checkpointing: saving state at step boundaries so crashes become interruptions, not disasters

The mechanism is simple to describe, even though the engineering that makes it work is not. After each tool call, each model inference, each subagent result, the full state of the run gets serialized and written to storage, keyed to a thread identifier. That thread ID works as a durable cursor: any worker, on any machine, can pick up that thread and rebuild exactly where the run stood, without needing to have been the worker that started it.

A checkpoint has to hold more than a step number. It needs the reasoning state at that point (the messages, the current plan, the active subtask), the tool outputs already received, the state of any subagents and their own checkpoints, and metadata needed to resume correctly, like retry counts and pending approvals.

LangGraph's checkpointing is a good concrete example of how this plays out. With a checkpointer configured, LangGraph saves the graph's state at every superstep. PostgreSQL is the default backend for production; the in-memory saver won't survive a restart. When a worker crashes, its lease on the run gets released, and a different worker can pick the thread up from the latest checkpoint without missing a beat.

Checkpointing at scale creates its own headache, though. Under a naive full-snapshot model, where the entire state gets rewritten at every step, storage grows roughly quadratically with the length of the run. A simulated 200-turn coding session under full snapshots produced 5,300 MB of checkpoint storage: a bill that makes long sessions expensive before they've done anything useful.

Deep Agents v0.6 fixed this with delta channels: store just the diff at each step, and write a full snapshot only periodically. The same 200-turn session dropped to 129 MB, a 41x reduction, with no extra configuration required. A snapshot_frequency setting, defaulting to every 50 steps, bounds how long resume takes, since a worker never has to replay more than 50 steps of deltas to rebuild state.

Checkpointing is the floor everything else stands on. Pause-and-resume, bounded memory, and retry policies all depend on durable state to return to.

Replay and idempotent tool calls: making reruns safe when the agent picks up mid-task

An agent calls a payments API, the worker crashes before it records that the call succeeded, and the naive fix reruns the step. The payments API gets hit a second time. Somebody's card gets charged twice. Nothing about that failure required a bad model. It required a rerun with no memory of what already happened, which is a much more common failure than people assume.

A tool call counts as idempotent when calling it more than once, with the same inputs, produces the same result and the same external effect as calling it exactly once. Getting there usually takes a few tactics combined, not one silver bullet.

Idempotency keys attach a deterministic identifier, built from the thread ID and step index, to each external call, and the receiving system deduplicates against that key. Check-then-act flips the order entirely: before executing a write, check whether the effect already exists (was the GitHub issue already filed?) and skip the write if it is. Read-only replay fits deterministic workflows best, where recovery replays only the read operations and skips any write whose output already sits in a checkpoint.

Agent-directed workflows complicate every one of these tactics, because the model might pick a different tool, or different arguments, the second time around. Replay logic has to account for a "same" step producing a genuinely different call on rerun. Read-only replay and check-then-act both handle that more gracefully than a blind retry ever could.

Platform support varies quite a bit here. Temporal offers workflow ID-based deduplication and deterministic replay at the workflow level, but it still expects developers to make individual Activities idempotent themselves, since Activities follow an at-least-once execution model by default. AWS's Durable Functions for Lambda, announced in December 2025, expose steps, waits, and checkpoints with retry semantics built for exactly this kind of problem. Cloudflare Workflows offers durable multi-step execution on top of Workers.

One question decides whether a tool belongs in a long-horizon agent: is this call safe to retry, and does it have a deduplication path? Say no to that, and the tool sets a hard ceiling on the whole agent's reliability, no matter how good the checkpointing sitting above it is.

Pause-and-resume and human-in-the-loop: suspending execution across unbounded time gaps

Without a real suspension mechanism, teams tend to build "wait for approval" as a polling loop: a worker stays alive and checks every few seconds whether a human has responded. That burns compute the entire time it waits, it breaks the moment someone deploys a new version and the worker gets recycled, and it has no good answer for a review that takes three days instead of three minutes.

True suspension works differently. The process releases its worker slot entirely, writes all pending state to the checkpoint store, and goes quiet until a signal wakes it back up. No polling, no held connection, no timeout ticking down in the background somewhere.

LangGraph's Deep Agents implementation gives two primitives for this. interrupt() pauses execution and surfaces a payload to whoever's watching, and it can sit anywhere in code, including inside a conditional or inside a tool itself, so the approval logic travels with the tool it's attached to instead of living somewhere separate. Command(resume=...) picks execution back up using whatever the human returned, and that return value can be any JSON-serializable object, not just a yes or no. A reviewer could hand back an edited draft, extra context, or a computed result the agent then works from directly. When multiple parallel branches each hit an interrupt(), they appear together in the state, and a single call can resume them at once or one at a time.

A related version of this occurs in tasks where the agent has to wait on something external to change rather than on a human. SentinelBench (arXiv 2606.05342) tests exactly this: an agent trying to buy concert tickets before they sell out can't make the on-sale time arrive faster by working harder. Acting more doesn't help here. Continuous polling is the wrong instinct; suspending until the environment actually changes is the right one. The benchmark contrasts a "sleep" agent, which polls at fixed intervals and burns resources doing it, against a "wait_for" agent, which suspends on a condition and only wakes once that condition is met, measuring both on task completion, reaction time, and resource use.

Pause-and-resume earns its place as a core primitive, not an optional add-on. A polling loop or a long-running HTTP connection can fake the behavior for a while. It stops faking it the second the worker restarts.

Bounded memory and context management: keeping long sessions from consuming themselves

Every step an agent takes adds to its message history. Tool outputs aren't small, either, since a full file, a raw API response, a page of search results, each eats a real chunk of context on its own. String enough of those together and the context window fills up, inference gets slower and more expensive, and the model's coherence starts slipping well before the window technically runs out of room.

Production harnesses handle this with three layers working together, not as competing options. History compression takes earlier stretches of conversation and reduces them to a summary, which replaces the raw messages in the active window. Tool output offloading moves large results, file contents, search dumps, diffs, out of the live context and into disk or an external store, leaving behind a reference or a short summary that expands on demand. Cross-session persistent memory handles what should outlive a single run entirely: facts, preferences, learned patterns, stored in a long-term memory layer instead of the context window that gets thrown away when the session ends.

A paper at ICLR 2026, MEM1, focuses directly on the tradeoff between memory consolidation and reasoning efficiency in long-horizon agents. Its existence says something on its own: the research community treats bounded memory as an engineering problem that needs solving head-on.

The clearest evidence for how much this matters is the same delta-channel number from earlier. Cutting 5,300 MB down to 129 MB for a 200-turn session decides whether that task length is practical or a nonstarter. Naive memory accumulation is a wall. It's a wall.

A few frameworks handle this differently, and the differences aren't cosmetic. Deep Agents, built on LangGraph, ships context management as a default: long threads get summarized, large tool results get offloaded to disk, and as of v0.6, storage is delta-backed out of the box. Letta builds stateful agents with long-term memory as a core feature, exposed through a REST API server. CrewAI's Flow persistence can save flow state across runs using SQLite by default.

None of this is free, and the tradeoff has to get made on purpose rather than left to whatever the defaults happen to be. Compress aggressively and cost drops, but detail goes with it. Retain aggressively and detail survives, but the context window fills up faster. Exploratory research usually wants breadth over precision. A long debugging session wants the opposite: precise local history, even if that means holding less overall.

Retry policies and backoff for recoverable versus escalating failures

Two mistakes occur constantly, and they sit at opposite ends of the same bad idea. Treat every failure as retryable, and permanent errors turn into runaway retry storms that never resolve. Treat every failure as fatal, and transient errors, the kind that would've cleared up on their own in a few seconds, end up killing tasks that had every chance of finishing.

Sorting failures into categories fixes most of this. Transient failures (rate limits, network timeouts, a service that's briefly down) call for retry with exponential backoff. Model errors (a hallucinated tool call, output that won't parse) might warrant a retry with a corrected prompt, or might need to escalate to a human, depending on how confident the system can be in the fix. Permanent failures (bad auth, a resource that genuinely doesn't exist, a policy violation) should escalate right away; retrying those just wastes time nobody has. Semantic failures are the trickiest kind: the step completed, nothing crashed, but the output is wrong anyway. Infrastructure alone can't catch that. It needs a verification layer sitting on top, checking output against some working notion of correctness.

Deep Agents sets retry policy per graph node: max attempts, backoff factor, and which exception types trigger a retry. That granularity affects how much of the retry budget gets spent where. A flaky external API can get a generous retry budget without wasting that same generosity on a deterministic computation step that should just fail fast and say so.

Backoff itself deserves more attention than it usually gets. Skipping it turns a fleet of agents all hitting the same rate-limited API at once into a thundering herd that makes the outage worse and longer, not shorter.

When retries run out, the run needs somewhere to go. Escalating through interrupt() to a human reviewer is one path. Checkpointing the state and parking the run for someone to look at later is another. Either way, the state has to be durable enough that waiting for a human doesn't mean losing everything that came before.