Articles

Fallback and controlled degradation

When a dependency fails, degrade the answer on purpose — or refuse, never pretend.

Reading: 4 minAutomation & AI

Article cover: Fallback and controlled degradation

A retry policy assumes the failure is transient and the same call will succeed. When a dependency is down, overloaded or refusing, that assumption is wrong, and repeating the call becomes part of the problem. This sheet is what follows: stop asking, and choose what to answer — stop calling the dependency, answer through another path, or answer with less, under one obligation: never answer as if nothing had happened.

The retry, timeout and loop control inside one call belong to the earlier sheet on retry, timeout and loop control; where a request goes in healthy operation belongs to the routing sheet. This sheet owns the failure path.

Stop asking: the circuit breaker

A circuit breaker “can prevent a caller service from retrying a call to another service (callee) when the call has previously caused repeated timeouts or failures”. It “acts as a proxy for operations that might fail”, sits between caller and callee, and trips when the callee is unavailable. Its reason inverts the retry policy: “an application shouldn’t continually retry an operation that’s unlikely to succeed. Instead, the application should quickly recognize the failed operation and handle the failure accordingly.” Such calls otherwise “hold critical system resources, such as memory, threads, and database connections” until they exhaust them.

The breaker “serves a different purpose than the Retry pattern”: retry expects eventual success, the breaker prevents a call likely to fail; when a fault turns out long-lasting, “it’s better for the application to prevent further requests going to the same resource and report a failure immediately”. Its state machine is closed (calls pass, failures counted), open (calls fail immediately) and half-open (a few trial requests); half-open “helps prevent a recovering service from suddenly being flooded with requests”.

Answer elsewhere: failover to another model or provider

OpenRouter’s models array “lets you automatically try other models if the primary model’s providers are down, rate-limited, or refuse to reply due to content moderation”. It is tried in order: “If the first model returns an error, OpenRouter will automatically try the next model in the list.” Any error triggers it by default: rate limits, downtime, context-length errors, moderation flags.

Failover is one more path, not a guarantee: “If the fallback model is down or returns an error, OpenRouter will return that error.” It changes the answer’s producer — requests are “priced using the model that was ultimately used” — and the list is finite: the fallbacks parameter “accepts at most 3 entries”.

Answer with less: degraded modes

Degradation is designed, not accidental. An application that invokes an operation through a breaker “might temporarily degrade its functionality, invoke an alternative operation to try to perform the same task or obtain the same data, or report the exception to the user and ask them to try again later.” In the open state, “rather than returning a failure and raising an exception, the Open state can return a default value that’s meaningful to the application” — and the worked example shows the shape: the application “gracefully degrades by returning default or cached responses”.

A degraded answer is a promise kept only if the substitute is the value chosen for this case and the substitution is visible to the user.

What must be refused instead of degraded

A fallback that fires on any error also fires where the error was correct. The default set includes context-length validation errors and moderation flags: a content refusal or an over-long request gets answered by a different model rather than returned. That is fine when the substitute is interchangeable. It is wrong when the model identity is part of the contract — a pinned version, a compliance boundary — or when the refusal exists on purpose. The breaker is no substitute either: you must not “use it as a substitute for handling exceptions in the business logic of your applications.”

The rule this sheet proposes, as framing: degrade the mode, not the guarantee. Refuse — return the error and say so — when the degraded value could be mistaken for a real result, when the substitute falls outside the model set allowed for the request, or when the fallback would silently route around a refusal a policy put there to hold.

The breaker is a decision, not a library

A breaker needs tuning: AWS warns that “circuit breakers introduce modal behavior into systems that can be difficult to test, and can introduce significant addition time to recovery”, so AWS also caps retries locally with a token bucket. Where a breaker is kept, it must match the dependency: “configure the circuit breaker to match the likely recovery pattern of the operation that it protects”, so that it neither raises errors after the fault is resolved nor drops back too quickly. A single breaker over several independent providers is the other trap: it can block a healthy shard alongside a failing one.

Level and prerequisites. L5 — specialist level: the reader must design a system’s failure path, not configure one, and decide which answers may still be given once a dependency is gone. Prerequisites: the L4 sheet on retry, timeout and loop control, and the L4 sheet on state, checkpoints and resuming.

Where to go next

References

  • Microsoft — Circuit Breaker pattern — the purpose of the breaker, its relation to Retry, the state machine, the degraded and default-value responses, and the tuning traps.
  • AWS Prescriptive Guidance — Circuit breaker pattern — the caller/callee intent of the pattern and its trip behaviour.
  • OpenRouter — Model Fallbacks — failover as an ordered model list, what triggers it, its limits and its billing.
  • Microsoft — Retry pattern — the point at which retrying stops and the breaker takes over.
  • AWS — Timeouts, retries, and backoff with jitter — the cost of breakers weighed against local throttling.
Nodrius Field Kit — cover

Get the Nodrius Field Kit

Enter your email and we'll send you the complete Field Kit: eight technical resources in one download.

Your email address is sent to Nodrius and used to deliver the Field Kit to you by email. Marketing messages are separate and optional: the checkbox does not affect your download, and you are only added to our updates list if you tick it. Privacy Policy.

Privacy & cookies

This site does not use cookies, analytics or tracking, and it does not profile you. There is nothing technical to switch off, so Accept and Reject change nothing about how the site works: either choice lets you browse everything normally.

The only difference is what your browser remembers: two entries in local storage — your choice here, so this notice is not shown again, and a marker that the Field Kit offer has already been made — so the offer is presented once and not on every page. They contain no identifier, they are not shared with anyone, and the site sets no cookie for either.

What we do with your data
How your email address is used when you ask for a Resource, how long it is kept and which rights you have — in the Privacy Policy.
What the site stores in your browser
Two entries in local storage — your choice on this notice and whether the Field Kit offer has been shown — and no cookies, no session storage, no third-party content: in the Cookie Policy.