Safety
What Virtual Trial will never do
The agents test and explain. Each promise below is enforced by code or by GitLab, not by asking the AI nicely, and links to what enforces it.
Never: write or edit dosing codeAgent boundaryWhat it is. A pipeline job that fails if any AI agent account committed a change to dosing code, the trial, or the gate limits. The only exception is an exact undo (git revert) of a commit already on the branch, verified by comparing the two patches.Why it matters. The rule does not depend on what an agent's instructions say. It is enforced by the pipeline, and tested against four ways to abuse the exception.
No agent has a tool that edits files. Independently, the agent boundary job fails any merge request where an agent account committed to lib/, gate/, or trial/. One exception, checked mechanically: the incident agent may undo a dosing commit that is already on the branch with git revert, because that restores code a trial and a person already approved. The job accepts it only if it is the exact inverse of that commit, and the undo still goes through the full trial and a person's merge.
Enforced by: Agent boundary check, Boundary tests, Incident flow toolset
Never: loosen a limit or skip the trialGate limitsWhat it is. The thresholds a change must stay within, in gate/thresholds.yaml. For example, time below 70 may rise by at most 0.5 percentage points.Why it matters. People own these numbers through code review (CODEOWNERS). Agents can never change them.
The gate limits live in gate/thresholds.yaml, owned by people through CODEOWNERS. The planning agent can only add the trial::full label, which makes the trial bigger, never smaller.
Enforced by: Gate limits, Code owners
Never: merge or ship a change on its ownMerge requestWhat it is. A proposed change to the code, reviewed before it is merged. GitLab's name for a pull request.Why it matters. Every dosing change arrives as one, so that is where the trial runs and where the verdict appears.
No agent has a merge, approve, or deploy tool. Changes merge only when a person merges the merge request; deploys run from the pipeline on protected branches.
Enforced by: Explain flow toolset, Deploy rules
Never: serve a broken engineCanary releaseWhat it is. A new engine version is deployed but receives no traffic until it passes a safety check on 151 dosing decisions. Then it gets all traffic and is checked again.Why it matters. A broken release never serves anyone. This caught a real crash on Oct 13: the bad version never received traffic.
A new engine version gets no traffic until it passes 151 decisions inside oref0's safety limits; after promotion it is checked again and rolled back on failure. This stopped a real crash on Oct 13.
Enforced by: Release gate
Never: touch a real patient or real dataVirtual patientWhat it is. A computer model of a person whose blood sugar rises with food and falls with insulin, the way a real body does. We use the UVA/Padova model via simglucose: 30 patients, children to adults.Why it matters. It lets us see what a code change would do to a body over a day, without risking anyone.Source
Every patient is simulated. No real glucose, pump, or health data is used or stored anywhere in the project.
Never: post to the upstream projectOpenAPS oref0What it is. The open-source algorithm that decides, every 5 minutes, how much insulin a pump should give, based on glucose readings and insulin already given. MIT licensed.Why it matters. It is the project Virtual Trial protects. Its logic was carried into other open-source dosing apps (AndroidAPS, Trio).Source
We do not open issues or pull requests on OpenAPS or other dosing projects. Maintainers hear from people, not bots.
Limits, stated plainly
- The virtual patients come from simglucose, an open-source implementation of a well-studied model. It is not the FDA-accepted commercial simulator and has no exercise model. UVA/Padova simulatorWhat it is. A mathematical model of how glucose and insulin move through the body, developed at the University of Virginia and the University of Padova.Why it matters. The FDA accepted the commercial version in 2008 as a substitute for animal trials in testing insulin controllers. We use an open-source implementation and do not claim FDA validation.Source
- Results compare the old and new code on the same patients and days. They are not predictions for any real person. Base and headWhat it is. Base is the dosing code before the change; head is the code with the change. The trial runs both on the same patients, meals, faults, and random seed.Why it matters. Because everything else is identical, any difference in outcomes comes from the code change.
- A passing trial means the change did not look riskier in our scenarios. It does not mean the change is safe. Safety gateWhat it is. The pipeline step that reads the trial's verdict and passes or blocks the change.Why it matters. It turns the trial into a decision: risky changes cannot merge or ship.
- Ten scenarios and a bounded search cannot cover every real day. Faults are modeled on reported failures, and new ones can be added to trial/scenarios.py. ScenariosWhat it is. Eight simulated days, each with meals and at most one fault: a normal day, a forgotten bolus, a large dinner, a sensor dropout, a false low, a sensor jump, a pump history that starts with a resume, and a bolus missing from history.Why it matters. Each fault is modeled on a failure reported in real dosing software or a recall, so the trial tests what has actually gone wrong before.
- This is a research and education project. It is not a medical device and makes no regulatory claims. IllustrationWhat it is. A drawn example, not output from a real trial run.Why it matters. We label it so nobody mistakes it for a result. Real results are on the Proof and For judges pages.
Research and education only. Not a medical device and not for therapy decisions.