Environments and test data: local, preview, staging and production without real personal data

Environments and test data are where good privacy intentions most often quietly fail. A company can have careful access controls, clear consent records and strict retention policies in production — and then a developer copies the production database to a laptop to reproduce a bug, and every one of those controls stops applying.

At Oryvelon we build and operate several independent companies, each with its own product and data. We run the same environment model in all of them: local, preview, staging and production, with production personal data allowed in production only. This article explains what each environment is for, how we build synthetic test data that is good enough to replace real data, and the habits that keep the boundary intact.

Why environments matter for privacy, not just engineering

Environments are usually discussed as an engineering concern: catching bugs before release, testing migrations, previewing designs. All true. But from a data point of view, every environment is also a place where data lives, and every place where data lives needs access control, secrets, logging and deletion.

If only production has real personal data, you only have to get those things right once, in the environment with the strongest controls. If staging, preview builds and laptops have copies too, you have to get them right four times — and you usually do not.

There is a simpler argument too. People trust a company with their data for the purpose of using its product. They did not agree to be a test fixture. For EduRelia, where the data concerns children, this is non-negotiable. For every other company, it is still the right default.

The four environments

Each Oryvelon company runs four kinds of environment. Smaller or earlier-stage companies may combine preview and staging at first, but the principles are the same.

Local

Local is where engineers write and run code on their own machine. It should be fast to set up, easy to reset and impossible to damage anything real from. A single command should create a local database and fill it with synthetic data.

Preview

Preview environments are short-lived builds, often one per change, used to review a feature before it is merged. Designers, product people and contractors can click through them. Because they are created often and shared widely, they must never contain real data or live credentials.

Staging

Staging is the dress rehearsal. It should match production as closely as practical — same configuration shape, same migrations, similar data volume — so that performance problems and migration failures show up before release. It uses a larger synthetic dataset and test-mode integrations.

Production

Production is the live product. It holds real data, uses live integrations and has the strictest access. Engineers do not work in production day to day; they deploy to it. Direct access to production data is rare, deliberate and logged, as described in Least-privilege access for small teams.

Separate secrets for every environment

An environment is only separate if its credentials are separate. Each environment in each company has its own database, its own storage and its own keys. Test-mode keys for payment providers, sandbox accounts for email platforms, low-limit development keys for model providers.

The rule that matters: a non-production secret must be useless against production. If a preview build's configuration leaks, the attacker should find synthetic data and sandbox services. We cover this in more depth in Secrets management basics.

This also prevents the classic accident — a script meant for local data that runs against production because someone had the live connection string in their environment.

The core rule: no production personal data in test

We state it plainly in every company's source-of-truth document: production personal data does not leave production. Not as a database dump, not as a CSV for a demo, not as "just a few rows" to reproduce a bug.

The rule has no casual exceptions, because casual exceptions are how the rule dies. The tempting cases, and what we do instead:

• "I need to reproduce a customer's bug." Reproduce the shape of the problem with synthetic data. Use logs, error codes and non-personal identifiers to understand what happened. If direct access is genuinely required, it happens in production, through a controlled break-glass process, not by copying the data out.
• "The demo looks empty." Build a better demo dataset. Realistic synthetic data is better for demos anyway, because it can be curated to show the product well.
• "We need realistic volume for a performance test." Generate it. Synthetic data can be produced at any scale.
• "We'll anonymise the copy." Anonymising real data is harder than it looks. Removing names and emails often leaves enough in addresses, timestamps, free text and combinations of fields to re-identify people. Generating data that was never real avoids the problem entirely.

What good synthetic test data looks like

The rule only survives if the alternative is good. Poor test data — five users called "Test User", three orders, no edge cases — pushes people back towards production copies. So we treat synthetic data as a product in its own right.

It is realistic in shape. Names, addresses, amounts, dates and text follow the formats and distributions of real data. A synthetic Shopify store for MerchNivo has products with variants, orders with discounts and partial refunds, returning and new customers, and several months of history.

It includes the edge cases. The most valuable test data is the data that breaks things: very long names, names in several scripts, zero-value orders, refunds larger than the original order due to a data error, missing fields, time zones around midnight, a school with one pupil, a talent profile with no photos. Every production bug that turns out to be data-shaped becomes a new synthetic case.

It is obviously fake. Synthetic emails use reserved example domains. Phone numbers use ranges that cannot reach anyone. Names can be realistic but should not be drawn from real customer lists. Nobody should ever wonder whether a record is a real person.

It is seeded automatically. A script creates the dataset from scratch. Local, preview and staging environments are populated by the same scripts, so everyone works from the same known state.

It is versioned with the code. When the schema changes, the seed data changes in the same change. Test data that falls out of step with the code is quickly abandoned.

It covers tenants. Every dataset includes at least two tenants, so cross-tenant isolation can be tested on every run. See Tenant isolation explained.

Synthetic data for each kind of company

Our companies are different businesses, and their test data reflects that. Some examples of what each company's synthetic dataset needs to show.

MerchNivo. Several synthetic stores with different sizes and product mixes. Seasonal patterns, a sudden spike in refunds for one product, a stockout, a discount code that is being overused, a store with almost no data. These let the team check that the operations signals fire correctly and that the daily briefing explains them accurately. The numbers in a briefing always come from store data; in test, that "store data" is synthetic but realistic.

CastLyra. Synthetic talent profiles at different verification stages, brands with different permissions, bookings in various states and contact-detail access that should and should not be granted. Profile images are generated or clearly placeholder, never real people's photos.

KeşifAtlası. Synthetic applicant profiles across nationalities, residence situations and document states, designed to exercise each branch of the verified rules database. When a rule changes, a synthetic profile that should flip from eligible to not eligible is added so the change can be tested before release. We describe the engine in Rules engines for eligibility.

EduRelia. Synthetic schools, classes and pupils with varied progress patterns. No real children, ever — including in demos for schools. The wider reasoning is in Data minimisation for children in edtech.

Noveniq. A test-mode store with synthetic products, customers and orders, plus sandbox payment and shipping integrations. Because Noveniq is also a real store we use as a testbed, the distinction matters: what we learn from the live store comes from production analytics and operations, handled under production rules; what developers test against is synthetic.

Testing AI features without real data

AI features raise the stakes, because testing them well needs a lot of realistic input, and prompts are easy to paste real data into.

We build evaluation sets from synthetic inputs: synthetic store snapshots for MerchNivo briefings, synthetic applicant scenarios for KeşifAtlası explanations, synthetic student questions for EduRelia. Each case has an expected structure and, where relevant, expected facts. These sets run whenever a prompt version or model route changes, as we describe in Prompt versioning and evaluation.

The staging configuration of our AI gateway uses separate keys and budgets from production. Logs from staging contain only synthetic content, so they can be inspected freely while debugging.

When production behaviour reveals a problem — an explanation that misreads a pattern, for example — we turn it into a synthetic evaluation case that reproduces the pattern, rather than copying the real input into the test set.

Promoting code, not data

A useful mental model: code moves up through environments; data does not.

Changes start locally, get a preview build, pass through staging and are deployed to production. Each step runs the same migrations and the same code. Data never travels the other way. Production data does not flow down to staging, preview or local.

Configuration moves up with the code, but secrets do not — each environment reads its own secrets from its own configuration. This keeps deployments repeatable and environments independent.

Access to each environment

Different environments call for different access. We scope them accordingly.

• Local belongs to the developer. Because it holds only synthetic data, a lost laptop is an inconvenience, not a data incident.
• Preview and staging are accessible to the team working on the company and to scoped contractors. Because they hold no real data, a contractor can be productive on day one without being granted anything sensitive.
• Production access is role-based, minimal and logged, per company.

This is one of the underrated benefits of synthetic data: it makes collaboration easier. A designer, a contractor or a new engineer can work in staging immediately, without an access request for personal data and without a risk review.

Rehearsing migrations in staging

Database migrations are one of the main reasons staging exists. A migration that adds a column in a second on a small local database may lock a large production table for minutes, or fail halfway on a row that violates a new constraint.

Staging catches most of these problems if its data is shaped like production. That is why staging's synthetic dataset is larger than local's and includes the awkward records: nulls where the schema used to allow them, very old rows created before a field existed, tenants with unusually large histories. We run every migration against staging first, time it and check that the application still works with both the old and new shape during deployment.

When staging passes a migration that later fails in production, the lesson goes back into the seed data. The failing pattern becomes a synthetic record, so the same class of problem cannot surprise us twice.

Resetting environments on purpose

Non-production environments get messy. Test accounts pile up, half-finished records linger, someone changes a setting by hand and forgets. Over time, staging stops matching the seed data and bugs appear that nobody can reproduce.

The fix is to make resets cheap and routine. Preview environments are created and destroyed with each change. Staging can be rebuilt from migrations and seed scripts on demand, and is reset on a regular rhythm. Local environments reset with one command. Because nothing in them is real, a reset loses nothing that matters, which is exactly the freedom synthetic data is meant to provide.

Handling support cases without copying data

Support is where the pressure to copy real data is strongest. A customer reports that their dashboard shows the wrong total, and the fastest route seems to be pulling their records into a test environment.

We take a different route. Support staff describe the problem using non-personal details: the tenant identifier, the feature, the time, the error code, the shape of the data ("an order with two partial refunds and a currency change"). An engineer recreates that shape with synthetic records in staging and fixes the bug there. If the investigation truly needs the real records, it happens inside production, through the support tooling or a logged break-glass session, and nothing is exported. For a MerchNivo merchant, this means their store's figures stay in production even while their bug is being fixed.

Common mistakes

• "Temporary" production dumps that live on a laptop or shared drive indefinitely.
• Staging connected to live integrations, so test orders trigger real emails or real payments.
• Shared credentials across environments, so staging can reach production storage.
• Stale seed data that no longer matches the schema, which pushes people back to real data.
• Screenshots of production in tickets and design files, exposing real customer details to anyone with access to the project tool.
• Test data with real-looking email addresses at real domains, which can send mail to strangers.
• Analytics and error tracking in test environments pointing at the same projects as production, mixing test and real data in reports.

A setup checklist for a new company

When we start a new company, part of the technical groundwork is setting up environments properly from the first week.

1. Create separate local, preview, staging and production configurations.
2. Create separate databases, storage and secrets for each.
3. Use sandbox or test-mode accounts for every third-party integration outside production.
4. Write the seed script before the first feature, with at least two tenants.
5. Add edge cases to the seed data as they are discovered.
6. Point analytics and error tracking in non-production to separate projects, or disable them.
7. Record the rule "production personal data does not leave production" in the source-of-truth document.
8. Restrict production access to named roles, with logging.

Doing this at the start takes a day or two. Retrofitting it after a year of production copies takes much longer, and some of those copies are never fully found.

Summary

A clean environment model gives each stage of development a clear job: local for building, preview for reviewing, staging for rehearsing and production for real customers. Each environment has its own secrets and its own data, and only production holds real personal data. That rule only survives if synthetic test data is good — realistic, full of edge cases, obviously fake, automatically seeded and versioned with the code — so we invest in it as seriously as in the product. Across Oryvelon's companies, from MerchNivo's synthetic stores to EduRelia's synthetic classrooms, this keeps customer data in the one place designed to protect it, and makes every other environment safe to share, break and rebuild.

Questions and answers

What is the difference between staging and production?

Production is the live environment that real customers use, with real data and live integrations. Staging is a production-like copy used to test releases before they go live, running on test data and test-mode integrations.

Why shouldn't production data be used for testing?

Copies of production data in test environments multiply the places personal data lives, usually with weaker access controls, logging and deletion. It also exposes customer data to people and tools that do not need it, and makes privacy promises harder to keep.

What is synthetic test data?

Synthetic test data is data generated to look and behave like real data without describing any real person or customer. It is created by scripts or fixtures, versioned with the code and designed to cover normal cases and edge cases.

Published by Oryvelon, a digital company builder · About the group · Our companies · More in Data, Privacy & Security

NextDomain, DNS and email security for a multi-brand group →