AI fallbacks and degraded modes: what a product does when the model fails

Every AI feature fails sometimes. A provider has an outage. A request times out at the worst moment. A model returns output that does not match the schema, twice. A product reaches its monthly budget on the 26th. None of this is exotic; it is ordinary operations for anything that depends on a model you do not run yourself.

The question is not whether a feature will fail but what the user sees when it does. At Oryvelon, every AI feature in every company ships with written AI fallbacks and a non-AI degraded mode. This note explains how we design them, why the degraded mode is usually more important than the fallback model, and what we learned about making failure boring.

Why AI features fail differently

Traditional software fails in fairly predictable ways. A database is up or down. An API returns an error code. You can usually tell what went wrong and when.

AI features add three failure types that are harder to spot.

Slow failure. The model responds, but after twenty seconds instead of three. Nothing is technically broken, and yet the user has left.

Quiet failure. The model responds quickly with confident output that is wrong, incomplete or outside the allowed format. Without validation, this looks exactly like success. See Structured outputs and schema validation.

Economic failure. The model works perfectly, but the product has spent its budget. Continuing to call it means losing money on every request. See Cost discipline for AI products.

A fallback design that only handles hard errors covers maybe a third of the real cases. Ours treats all four — hard errors, slowness, invalid output and budget exhaustion — as triggers for the same planned behaviour.

The principle that makes degraded modes possible

Degraded modes are easy to design if you follow one rule from the start: the model is never the source of the facts.

In our products, verified systems supply the facts and the model explains them. MerchNivo's numbers come from the Shopify store. KeşifAtlası's eligibility outcomes come from a verified rules database. CastLyra's verification status comes from its own review process. The model turns those facts into plain language, suggestions or a structured draft. We wrote about this in AI explains, verified data decides.

The consequence for failure handling is enormous. If the model disappears, the facts are still there. The degraded mode can show them directly — as a table, a checklist, a status badge, a template — and the user still gets what they actually came for. They lose the narrative, not the truth.

Products that let the model generate the facts have no such option. When the model fails, there is nothing to fall back to except an error message.

Anatomy of a fallback chain

For each AI feature, the product's configuration in our AI gateway defines a fallback chain. A typical chain has four steps.

Two details matter more than the steps themselves.

First, the whole chain has a single deadline. If a feature should respond within a few seconds, the chain does not get a few seconds per step. It gets a few seconds in total, and each step only runs if enough time remains. Otherwise a chain of three careful fallbacks turns into a thirty-second wait, which is just a slow failure with extra steps.

Second, the alternative model must be approved for that feature. It is tempting to say "if the main model fails, use any other one". We do not. A model that has not passed the feature's evaluation set can produce output that is worse in ways validation will not catch: a different tone, weaker adherence to content boundaries, subtle factual drift. For some features we do not configure an alternative model at all and go straight from retry to degraded mode. See Model routing and Prompt versioning and evaluation.

Designing the degraded mode first

We now design the non-AI version of a feature before, or alongside, the AI version. That sounds backwards. It turns out to be one of the most useful habits we have.

Designing the degraded mode forces three questions early.

1. What is the user's actual job here? If the degraded version still does the job, the AI is a quality layer on top. If it cannot, the feature may depend on the model in ways that are risky.
2. Which facts does the feature need? Listing them makes it obvious which verified system provides each one, and exposes any fact that only the model was producing.
3. What does "good enough" look like without AI? Often the answer is simpler and more useful than expected, and sometimes it becomes the default for certain users.

A degraded mode is not an error state with a nicer icon. It is a real, maintained piece of the product with its own design, its own copy and its own tests.

Worked examples across the group

MerchNivo: the briefing becomes a table

MerchNivo is an AI e-commerce employee for Shopify stores. Its best-known feature is the morning briefing: a short, plain-language summary of what happened in the store and what deserves attention.

Imagine a store that had an unusual spike in refunds overnight — six refund requests, four of them for the same product. On a normal day, the briefing says something like: refunds are higher than usual, most relate to one product, you may want to check the listing and recent reviews.

On a day when the model is unavailable, the merchant opens MerchNivo and sees the degraded briefing. The same signals appear as a compact table: refunds (6, above the recent range), top refunded product (4 of 6), low-stock items, abandoned checkouts. The operations signal that flagged the refund spike is a rule, not a model output, so it still fires. A small note says the written summary is temporarily unavailable.

The merchant loses the prose and the suggested wording of a next step. They do not lose the fact that something is wrong with one product. That is the difference between a degraded mode and an outage.

KeşifAtlası: the rules still answer

KeşifAtlası helps people understand visa and relocation eligibility, grounded in a verified rules database. The AI explains the rules; it never decides.

If the explanation model fails during a free eligibility test, the rules engine has already produced the outcome and the list of rules that applied. The degraded mode shows that outcome with the rule texts themselves and their official wording, plus a clear route to ask a person. It reads more formally, but it is complete and correct.

For the paid report the bar is higher. We do not want a customer who paid for a clear, explained report to receive a rule list. So if generation fails for a paid report, the report is queued, the customer is told it will arrive shortly, and the job is retried in the background. If it still fails, a person reviews it. Degraded mode for a paid deliverable is often "later, and checked", not "now, but plainer". See From free test to paid report.

ZodiVela: fewer words, same boundaries

ZodiVela offers personal discovery readings framed for reflection and entertainment. Its degraded mode uses pre-written, reviewed content blocks for the relevant sign and theme, assembled by simple rules. They are less personal than a generated reading but fully within the product's content boundaries, because every block was written and reviewed in advance.

One rule here is firm: if a user paid credits for a personalised reading and received the degraded version, the credits are not consumed. Charging full price for a fallback would turn a technical failure into a trust problem.

EduRelia: stop, don't improvise

EduRelia serves students, families and schools. For a learning product used by children, a wrong or inappropriate answer is worse than no answer. Its fallback chain is deliberately short: one retry, then the degraded mode, which points the student to the relevant lesson material and suggests asking a teacher or parent. There is no approved alternative model for the tutoring features unless it has passed the same age-appropriate content evaluation. See Safety boundaries for consumer AI products.

Budget limits are a fallback trigger too

Every product has a monthly AI budget with a soft limit and a hard limit. The soft limit sends an alert to the company owner. The hard limit switches the product, or specific features, into degraded mode.

The alternative approaches are both worse. Letting spending continue past the limit turns an operations issue into a financial one. Returning errors when the budget is hit punishes users for something they cannot see or influence.

In practice, hitting a hard limit rarely means switching everything off. Budgets are usually set per feature, so the least essential features degrade first. A product might keep its core explanation feature running while pausing optional extras such as alternative phrasings or long-form expansions. Heavy users who have hit a per-user allowance may see the degraded mode while everyone else continues normally. Each of those choices is written down in the product's configuration, not decided under pressure at 11pm.

When a product hits its hard limit, it is also a signal for the monthly review. Was the budget wrong, or did usage change, or is a feature more expensive than it is worth? That feeds the continue, stop or scale checkpoints we run for every company.

What the user should see

The copy for a degraded mode deserves as much care as the copy for the main feature. A few rules we follow:

• Say what is missing, not what broke. "The written summary is not available right now" is more useful than "AI service error".
• Show what still works. Lead with the facts the user can rely on.
• Do not overpromise. Avoid "back in 5 minutes" unless you know it. "We'll try again automatically" is fine if true.
• Do not charge for what was not delivered. Credits, usage allowances and paid reports should reflect what the user actually received.
• Offer a human route where the stakes are high. For KeşifAtlası and EduRelia, the degraded mode always includes a way to reach a person. See Human escalation in AI products.

Users are remarkably tolerant of a plainer experience when it is honest. They are much less tolerant of spinners that never finish.

Observability: know when you are degraded

A fallback that works silently has a hidden cost: nobody notices that the product has been in degraded mode for three days. The gateway records every step of the chain per product and per feature — how many requests succeeded on first try, how many used the retry, how many used the alternative, how many ended in the degraded mode.

We watch the fallback rate as a first-class health metric. A small, steady rate is normal. A sudden jump usually means a provider issue. A slow climb usually means something on our side has drifted: a prompt change that increases invalid outputs, an input format that changed, a model update that altered behaviour.

We also log the reason for each fallback in categories — timeout, provider error, schema failure, safety block, budget limit — so that the weekly health review can tell the difference between "the provider had a bad afternoon" and "our new prompt fails validation one time in ten".

Test the fallback, not just the feature

The most common fallback bug is that the fallback itself does not work. It was written once, never exercised, and broke quietly when something else changed.

So we test fallbacks deliberately:

• In automated tests, every AI feature has a test that forces each step of the chain — simulated timeout, simulated invalid output, simulated budget limit — and checks what the user sees.
• In staging, we can switch a feature to degraded mode with a configuration flag and click through it like a user would.
• Before launch, the company owner looks at the degraded mode and answers one question: would I be comfortable if a paying customer saw only this for a day?

If the honest answer is no, the degraded mode is not finished.

Common mistakes

A few patterns we have seen, in our own early work and elsewhere:

Falling back to a bigger model. When a small model fails, reaching for a larger one feels safe. It is often slower, far more expensive and may not have been evaluated for the task. It can also hide a prompt problem that should be fixed.

Unlimited retries. Retries multiply cost and latency. A single retry catches most transient issues; after that, move on.

Caching a fallback as if it were the real answer. If a degraded response is stored in a cache under the same key as a full response, users can keep seeing the degraded version long after the model has recovered. Degraded output should be marked and cached separately, if at all. See Latency and caching for AI.

One global switch. A single "AI off" switch for a whole product is useful in an emergency, but it should not be the only control. Per-feature fallbacks let the least important features degrade while the core keeps working.

Hiding the degraded state from the team. If only the gateway knows, nobody fixes the cause.

A fallback checklist for new AI features

Before any AI feature ships in an Oryvelon company, its entry in the company's source-of-truth document answers these questions:

1. What is the user's job, and does the degraded mode still do it?
2. Which verified systems provide the facts, and do any facts come only from the model?
3. What is the total time budget for the feature, including fallbacks?
4. Is there an approved alternative model or prompt, and which evaluation did it pass?
5. What exactly does the user see in degraded mode, and has someone reviewed that copy?
6. What happens to credits, allowances or payments if the degraded mode is used?
7. When does budget exhaustion trigger degradation, and for which features first?
8. How is the fallback rate monitored, and who is alerted?
9. Has each step of the chain been forced in a test?

Nine questions, usually answered in a page. It is one of the cheapest reliability investments we make.

Summary

AI fallbacks and degraded modes are what separate a product that occasionally feels plainer from one that occasionally breaks. The design becomes straightforward once the model is never the source of the facts: when it fails, verified data can still be shown directly. We give every feature a short, time-bounded fallback chain of retry, approved alternative and non-AI degraded mode; treat slowness, invalid output and budget limits as failures alongside hard errors; write honest copy that never charges for what was not delivered; and monitor and test the fallback as carefully as the feature itself.

Questions and answers

What is a degraded mode in an AI product?

A degraded mode is a planned, simpler version of a feature that works without the AI model, such as showing verified data in a table instead of a generated summary. It keeps the product useful when the model is unavailable, too slow or over budget.

What should an AI fallback chain include?

A typical chain is a limited retry, then an alternative model or simpler prompt, then a non-AI degraded mode, and finally a clear message to the user. Each step has a time limit so the whole chain stays within an acceptable response time.

Should an AI product switch to another model automatically when the main one fails?

Only to models that have been evaluated for that specific feature and approved in its configuration. Switching to an untested model can produce worse or unsafe output, which is often more harmful than showing a plain degraded mode.

Published by Oryvelon, a digital company builder · About the group · Our companies · More in AI Operations

NextLatency and caching for AI features: keys, batching and streaming →