Articles

Retry, timeout and loop control in agent workflows

Timeouts, retries and backoff bound an agent loop before one failure becomes an outage.

Reading: 4 minAutomation & AI

Article cover: Retry, timeout and loop control in agent workflows

An agent workflow is a loop: the model chooses an action, the layer below executes it, the result returns, and the model chooses again. Any iteration can fail — a model call that never answers, a tool briefly down, a database throttling. This sheet covers the failure inside one iteration: what a timeout decides, what may be retried, and how repetition is bounded before a fault becomes an outage. The loop’s shape and stopping conditions belong to the companion sheet on the action loop; the obligations of a single request belong to the earlier sheet on its timeout, retry and idempotency, and this sheet applies both across the loop’s layers.

A loop multiplies a single failure

One failed request is one; one agent iteration is many, each a call that can time out. Retrying is not free — “It’s not always safe to retry. A retry can increase the load on the system being called, if the system is already failing because it’s approaching an overload.” When every layer retries independently the retries multiply, which is why “our best practice is to retry at a single point in the stack.” Microsoft states the same trap from the other side: “if a task that contains a retry policy invokes another task that also contains a retry policy, this extra layer of retries can add long delays to the processing.” Decide which layer owns the retry before the loop runs.

Timeouts: one per call, one for the run

A loop needs two timers. The per-call rule is older than LLM APIs: “A best practice in Amazon is to set a timeout on any remote call, and generally on any call across processes even on the same box.” HTTP names the states a loop must expect: 429 means the caller sent “too many requests in a given amount of time”, and 503 marks a server “currently unable to handle the request due to a temporary overload or scheduled maintenance”. The second timer is the run’s own wall-clock budget — a provider can enforce one, since the Anthropic SDKs “validate that your non-streaming Messages API requests are not expected to exceed a 10-minute timeout”. Per-call timeouts bound a step; only a total budget bounds a run.

Which failures the loop may retry

A timeout is not even a verdict: “A timeout or failure doesn’t necessarily mean that side effects haven’t happened.” The transient class may be retried — “the momentary loss of network connectivity to components and services, the temporary unavailability of a service, or timeouts that occur when a service is busy” — because such faults are “typically self-correcting, and if the action that triggered a fault is repeated after a suitable delay it’s likely to be successful”. A client should not retry a call “unless it has some means to know that the request semantics are actually idempotent”, and a tool that charges a card is not. When a service answers 429, the response “MAY include a Retry-After header indicating how long to wait before making a new request” — the earliest moment to ask again.

Backoff, jitter and a retry budget

For failures worth retrying, the wait is part of the policy. Backoff increases the delay between attempts, and because “Exponential backoff can lead to very long backoff times”, implementations cap it. A capped backoff makes every client retry at the same constant rate, so implementations add jitter to the wait — “Jitter adds some amount of randomness to the backoff to spread the retries around in time” — and also “limit the number of times that the client retries”; AWS adds “limiting retries locally using a token bucket” that lets calls retry freely while tokens last and then “retry at a fixed rate when the tokens are exhausted”. That bucket is the retry budget, above any single call. The stopping rule is not “until it works”: “We stop retrying when the retries are not helping to improve availability.”

Loop control: stop the loop, not only the call

Two retries live in one agent workflow. One repeats a failed call, under the policy above. The other is the model choosing the same failing action again on the next turn — a retry decided by the LLM, with no backoff and no budget, which only loop control bounds. In practice that is a bounded attempt count per call, a retry budget for the run, and an iteration limit that stops a model re-issuing the same broken tool. Where an operation is not worth repeating, do not repeat it — “For some noncritical operations, it’s better to fail fast rather than retry several times and affect the throughput of the application.” Check the layer underneath first: the Anthropic SDK “automatically retries transient failures (such as connection errors, rate limits, and 5xx server errors) with exponential backoff, twice by default”, so a hand-rolled retry above the SDK stacks a second policy on a call the SDK already retries.

Level and prerequisites. L4 — design level: the reader must place retry, timeout and budget controls across an agent’s layers and say which layer owns the retry, not merely configure a library. No numbers are prescribed — none of the sources sets an attempt count, timeout or budget — because they belong to the pipeline. Prerequisites: the companion sheet on the action loop and its stopping conditions, and the earlier treatment of a single request’s timeout, retry and idempotency obligations.

Where to go next

References

  • AWS — Timeouts, retries, and backoff with jitter — the timeout rule, the amplification of nested retries, capped backoff, the token-bucket retry budget, and when to stop.
  • Microsoft — Retry pattern — transient faults as self-correcting, fail-fast, and the trap of nested retry policies.
  • IETF — RFC 9110 — the client rule for when a call may be retried, and the 503 status definition.
  • IETF — RFC 6585 — the 429 status code and its Retry-After guidance.
  • Anthropic — Errors — the SDK’s default retries and the 10-minute request ceiling.
Nodrius Field Kit — cover

Get the Nodrius Field Kit

Enter your email and we'll send you the complete Field Kit: eight technical resources in one download.

Your email address is sent to Nodrius and used to deliver the Field Kit to you by email. Marketing messages are separate and optional: the checkbox does not affect your download, and you are only added to our updates list if you tick it. Privacy Policy.

Privacy & cookies

This site does not use cookies, analytics or tracking, and it does not profile you. There is nothing technical to switch off, so Accept and Reject change nothing about how the site works: either choice lets you browse everything normally.

The only difference is what your browser remembers: two entries in local storage — your choice here, so this notice is not shown again, and a marker that the Field Kit offer has already been made — so the offer is presented once and not on every page. They contain no identifier, they are not shared with anyone, and the site sets no cookie for either.

What we do with your data
How your email address is used when you ask for a Resource, how long it is kept and which rights you have — in the Privacy Policy.
What the site stores in your browser
Two entries in local storage — your choice on this notice and whether the Field Kit offer has been shown — and no cookies, no session storage, no third-party content: in the Cookie Policy.