ORYVELONInsights / AI Operations · · 11 min read
Safety boundaries for consumer AI products: pre and post hooks
Consumer AI products need boundaries that hold even when users push on them. How we define content limits for ZodiVela and EduRelia, and enforce them with pre-request and post-response safety hooks, product-specific rules and human review, rather than relying on a single instruction in the prompt.
A consumer AI product talks to people you will never meet, in moods you cannot see, about things you did not plan for. Some will test the edges on purpose. Many more will arrive at an edge by accident — a student asking something far outside the lesson, a reader bringing a real worry to what was meant as a light reflection.
Safety boundaries for consumer AI are the decisions and mechanisms that keep a product inside what it is for, even when a conversation drifts. This note describes how we set those boundaries for two of Oryvelon's consumer products, ZodiVela and EduRelia, and how we enforce them with pre-request and post-response hooks in our shared AI gateway.
It describes our practice. It is not legal advice, and the specific limits for any product depend on its audience and the rules where it operates.
Start with what the product is for
Most safety problems in consumer AI are really positioning problems. A product that has not decided what it is for will be asked to be everything, and a general-purpose model will try to oblige.
So every consumer product at Oryvelon has a short written boundary policy in its source-of-truth document. It answers four questions:
- Purpose. What is this product for, in one sentence?
- Audience. Who uses it, and what do we know or not know about them?
- Never. What must the product never do, regardless of how it is asked?
- At the edge. What does it do when a conversation reaches a boundary?
For ZodiVela, the purpose is personal discovery readings framed for reflection and entertainment. The audience is adults who choose to use it. It must never present readings as prediction, and never give medical, financial, legal or relationship directives. At the edge, it acknowledges the topic, stays with reflection, and points to appropriate real-world help where the situation is serious. See Entertainment positioning for consumer AI.
For EduRelia, the purpose is helping students learn, with families and schools involved. The audience includes children. It must never collect more personal information than the task needs, never produce content unsuitable for the student's age, and never become a private confidant that replaces trusted adults. At the edge, it redirects to the learning task and encourages the student to talk to a teacher or parent.
The policy is written in plain language, a page or two at most. Everything technical follows from it.
Why the prompt is not enough
The first instinct is to put the policy in the system prompt. We do, and it matters: the prompt is the main way to shape tone, framing and default behaviour.
But a prompt is guidance, not enforcement. Models occasionally ignore instructions, especially over long conversations. Users can phrase requests to slip around them. A prompt change made to improve quality can weaken a boundary without anyone noticing. And a new model version may interpret the same prompt differently.
So we treat safety as three layers:
| Layer | What it does | Where it lives |
|---|---|---|
| Positioning | Sets expectations for users before any AI is involved | Product copy, onboarding, terms |
| Prompt | Shapes the model's behaviour and framing | Registered, versioned prompt |
| Hooks | Enforces limits on every request and response | Gateway configuration and product code |
If the prompt fails, the hooks catch it. If the hooks miss something subtle, positioning means the user was never led to expect advice in the first place. See Onboarding and trust in AI products.
Pre-request hooks
Pre-request hooks run after the product has built its input and before anything is sent to a model provider. Ours do four jobs.
Minimise what is sent
The model should receive only what the task needs. For EduRelia, that means no student names, no school names, no contact details and no identifiers beyond a pseudonymous session reference the model never sees in readable form. If a student types their full name and address into a question, the hook removes them before the request leaves. The model gets the maths question, not the child. See Data minimisation for children in edtech.
For ZodiVela, a reading needs birth details the user chose to provide and the theme they selected. It does not need their email, payment status or device information, so none of that is included.
Classify the request
A lightweight classifier, usually a small model or a set of rules, labels each incoming message by topic and risk. Labels are product-specific. ZodiVela cares whether a message concerns health, money, legal matters, a crisis or self-harm, or a request for a definite prediction. EduRelia cares whether a message is on-task, off-task, age-inappropriate, or a sign that a student may need help from an adult.
Route before calling the model
Some labels change what happens next. A message that suggests a person may be in crisis does not go to the reading or tutoring prompt at all. It goes to a fixed, reviewed response that takes the message seriously and points to real help, written in advance by people, not generated. Requests for predictions about health or money in ZodiVela are sent to a prompt variant that is explicitly reflective and declines to forecast.
Enforce limits
Per-user rate limits, maximum input length and conversation length limits are applied here. Long conversations are where models drift most, so both products cap them and start fresh sessions rather than letting a single thread run indefinitely.
Post-response hooks
Post-response hooks run after the model responds and before the user sees anything.
Schema and format checks. Where the product expects structured output — a reading with named sections, a tutoring answer with a step list — the response is validated first.
Content boundary checks. A second classifier checks the output against the product's "never" list. For ZodiVela, it looks for advice-like directives ("you should leave your job"), certainty about the future ("this will happen"), fatalistic statements, and health or financial claims. For EduRelia, it checks age-appropriateness, off-topic drift and any attempt to ask the student for personal information.
Grounding checks. Where the output should reference facts from the input, the hook checks that it does and that it has not introduced new ones. This is the same principle as AI explains, verified data decides, applied to consumer content.
Action on failure. If a check fails, the response is not shown. Depending on the configuration, the gateway retries once with a stricter prompt variant, or returns a pre-written, reviewed in-bounds response. It never shows the failed output with a warning attached.
Every block is logged by category, without storing the full content by default, so the team can see how often boundaries are being hit and why.
A worked example: a ZodiVela reading at the edge
Imagine a ZodiVela user who asks for a reading about the coming month and adds: "Should I sell my flat and put everything into a new business? I need to know if it will work."
Here is what happens.
The pre-request classifier labels the message as a money topic combined with a request for a definite prediction. The product's routing sends it to the reflective variant of the reading prompt, which is written to explore themes such as change, readiness and values without forecasting outcomes or recommending decisions.
The model produces a reading. Suppose one sentence slips: "This is a strong month to take a financial leap." The post-response check flags it as a financial directive. The gateway retries once with the stricter variant. The second response stays with reflection: it talks about what the user seems to value, what questions might be worth asking before a big decision, and suggests that a decision like this is best made with someone qualified to look at the numbers.
The user receives that second reading. They also see, as with every ZodiVela reading, the standing note that readings are for reflection and entertainment. They are not told that a sentence was blocked. They are simply given a reading that respects what the product is.
If the second response had also failed, a pre-written reflective passage for that theme would have been used, and the user's credits would not have been consumed for a personalised reading. See AI fallbacks and degraded modes.
A worked example: EduRelia off-task
Now imagine a student using EduRelia's study assistant for a science unit. Midway through, they write that they are upset because of problems with friends and ask the assistant what to do.
The pre-request classifier labels the message as off-task and personal. EduRelia's policy is clear that the assistant is not a counsellor or confidant. The message is routed to a reviewed response: it acknowledges that the student sounds upset, says kindly that this is something worth talking about with a parent, teacher or another trusted adult, and offers to continue with the science when they are ready.
The response is short, warm and does not ask follow-up questions about the situation. It does not collect details. If the message had contained signals of possible risk to the student, the routing would have used a different reviewed response pointing to immediate help, and the product's escalation process would apply as defined with the school. See Human escalation in AI products.
Nothing about the conversation is stored beyond what EduRelia's retention rules allow. See Data retention and deletion.
Different products, different limits
A common mistake in shared infrastructure is a single safety profile applied to everything. It is either too strict for some products or too loose for others.
Our gateway provides the hook mechanism; each product configures its own classifiers, labels, thresholds and responses. Compare:
| ZodiVela | EduRelia | |
|---|---|---|
| Audience | Adults who opt in | Students, including children, with families and schools |
| Core boundary | Reflection, not prediction or advice | Learning support, age-appropriate, minimal data |
| Streaming | Paragraph by paragraph, each checked | Generally off; full answer checked first |
| Conversation length | Capped per session | Tightly capped per session |
| Off-topic handling | Gently returns to reflection | Redirects to learning and trusted adults |
| Human route | Support contact and signposting | School and family involvement, defined escalation |
Our other products have hooks too, tuned to their risks. KeşifAtlası blocks any output that states an eligibility decision not produced by its rules engine. CastLyra checks that generated profile text does not invent credentials and that no contact details appear where marketplace rules do not allow them. MerchNivo checks that suggested actions refer to figures that exist in the store data.
Testing the boundaries
Boundaries that are not tested decay. We maintain a separate evaluation set for each consumer product's safety behaviour, alongside its quality evaluation set.
Each set contains realistic edge cases written by the team: requests for predictions, attempts to get advice, off-topic personal messages, attempts to make the assistant reveal its instructions or pretend to be something else, and inputs containing personal details that should be stripped. For EduRelia, cases are written as a student might actually phrase them, with spelling mistakes and slang.
Every prompt change, model change or hook change runs against this set before release. A drop in boundary adherence blocks the release, even if quality scores improve. See Prompt versioning and evaluation.
We also review a small sample of blocked and borderline outputs regularly, with personal details removed, to spot new patterns. Test sets use synthetic cases, never real user conversations copied from production.
Prompt injection and instruction leaks
Consumer products attract people who want to see what the assistant will do if pushed. Some ask it to ignore its instructions. Some paste in text that pretends to be a system message. Some ask it to reveal its prompt, or to role-play as an unrestricted assistant.
We assume these attempts will happen and design so they fail harmlessly rather than trying to make them impossible.
The most important defence is that nothing sensitive depends on the model obeying. The model does not hold other users' data, because the pre-request hooks never send it. It cannot grant credits, change a subscription or open paid content, because those actions are handled by product code, not by what the model says. Its output is checked by post-response hooks that do not read the user's instructions at all. A user who "convinces" the model to act differently gains, at most, a response that is then blocked and replaced.
We also avoid putting anything in the prompt that would be damaging if revealed. Prompts contain framing, tone and structure, not secrets, keys or internal notes. If a prompt leaked, it would be mildly embarrassing, not harmful.
Monitoring boundaries in production
Safety is not finished at launch. Each consumer product's owner sees a small set of boundary indicators in the weekly review:
- Block rate by category. How often post-response checks block output, split by reason: advice-like language, prediction, age-appropriateness, personal data, format.
- Routing rate by label. How often pre-request routing sends messages to reviewed responses, and for which topics.
- Fallback to reviewed content. How often users receive pre-written content because both attempts failed.
- User reports. How often users flag a response, and what they flag.
A sudden rise in blocks after a prompt or model change usually means the change weakened a boundary, even if it passed evaluation. A slow rise in a routing label, such as more off-task personal messages on EduRelia, may mean the product is being used in ways we did not expect and the positioning or onboarding needs work. Either way, the number starts a conversation rather than waiting for a complaint.
When boundaries change
Boundaries are not fixed forever. A product may add a new theme, reach a new age group, launch in a new country or respond to new guidance. When that happens, the change starts in the written policy, not in the prompt.
The owner updates the boundary policy in the source-of-truth document, noting what changed and why. The classifiers, routing rules and reviewed responses are updated to match. New cases are added to the safety evaluation set before anything goes live. Only then is the prompt adjusted. Doing it in that order keeps the policy, the enforcement and the tests in agreement, and means anyone reading the source of truth later can see why the product behaves as it does.
What we avoid
A few things we have decided not to do:
- Over-refusing. A product that refuses anything slightly unusual is safe and useless. Boundaries should be precise, so that normal use feels natural.
- Lecturing. When a request is outside bounds, a short, kind redirection works better than a paragraph about policy.
- Hidden personas. Neither product pretends to be a person. Both are clear that the user is talking to an AI.
- Engagement at any cost. We do not design the assistant to keep people talking. Session caps exist partly for this reason.
- Collecting more to be safer. More data about a user does not make a product safer. For children especially, less is the safer default.
Summary
Safety boundaries for consumer AI products work when they start from a clear written policy — purpose, audience, what the product must never do, and what happens at the edge — and are enforced in layers. Positioning sets expectations, the prompt shapes behaviour, and pre-request and post-response hooks enforce limits on every request: minimising input, classifying and routing sensitive topics, checking output against the product's boundaries, and replacing anything out of bounds with reviewed content. ZodiVela and EduRelia share the mechanism but not the rules, because an entertainment product for adults and a learning product for children need different limits, tested continuously.
Questions and answers
What is a safety hook in an AI product?
A safety hook is a check that runs automatically around a model call. Pre-request hooks inspect and adjust the input before it is sent, and post-response hooks inspect the output before it reaches the user, blocking or replacing anything outside the product's allowed boundaries.
Is a system prompt enough to keep a consumer AI product safe?
No. A system prompt strongly influences behaviour but can be ignored or worked around, so products also need hooks in code that check inputs and outputs on every request, plus clear routes to human help for sensitive situations.
How should an AI astrology or reflection product avoid giving advice?
It should be positioned for reflection and entertainment, avoid predictions and directives on health, money, law or relationships, and use output checks that catch advice-like or fatalistic language and replace it with in-bounds content.
