Research prototype on GitLab and Google Cloud
A clinical trial for every change to insulin dosing code.
Before a code change can reach an insulin pump, Virtual Trial runs the old and new code on the same virtual patientsVirtual patientWhat it is. A computer model of a person whose blood sugar rises with food and falls with insulin, the way a real body does. We use the UVA/Padova model via simglucose: 30 patients, children to adults.Why it matters. It lets us see what a code change would do to a body over a day, without risking anyone.Source, through the same bad days, and blocks the change if dangerous blood sugar becomes more likely.
New here? Every button explains a word or a number in plain language. Hover, tap, or tab to it.
Blocked In the full trial this change made low blood sugar risk rise in most matched runs. A person reads the explanation; the change does not ship.
Dosing software can hurt people
People with type 1 diabetesType 1 diabetesWhat it is. A condition where the body makes no insulin, so people must give themselves insulin every day for life.Why it matters. About 9.5 million people live with it worldwide; many rely on pumps and software to dose.Source trust software to decide their insulin every few minutes. When that software changes, a small mistake can become a dangerous dose.
Today's tests check outputs, not bodies
Open-source dosing code like OpenAPS oref0OpenAPS oref0What it is. The open-source algorithm that decides, every 5 minutes, how much insulin a pump should give, based on glucose readings and insulin already given. MIT licensed.Why it matters. It is the project Virtual Trial protects. Its logic was carried into other open-source dosing apps (AndroidAPS, Trio).Source is tested with open-loop testsOpen-loop testingWhat it is. Feeding fixed inputs to the algorithm and comparing its output with an expected answer, with no body in between.Why it matters. It catches typos but cannot show harm: a dose that looks reasonable once can stack up into a dangerous low over hours.: a fixed input, an expected output. That catches typos. It does not show what a change does to a body over a day, or what happens when real life gets messy. Its own tests already failoref0's own tests already failWhat it is. Run as published, 9 of oref0's 243 tests fail on its current development branch (7 on the 0.7.1 release), because the code changed and the tests did not.Why it matters. Before Virtual Trial these failures were ignored, so a new failure would have been ignored too. Now any new failure blocks the change., and nothing stopped that.
Virtual Trial adds the missing check to every change: a closed loopClosed loopWhat it is. The algorithm doses, the virtual body responds, the next reading feeds the next decision, every 5 minutes.Why it matters. Mistakes compound over hours. Checking one decision at a time misses that. with virtual patients and injected faultsFault injectionWhat it is. Deliberately adding real-life problems to the trial: sensor dropouts, false readings, missing pump records, forgotten boluses.Why it matters. Many real dosing bugs only appear when data is messy, not on a perfect day. modeled on real bugs and recalls.
Both directionsHigh blood sugar riskWhat it is. How long glucose stays far above range (above 250 mg/dL), a warning sign for diabetic ketoacidosis.Why it matters. A gate that only checks lows would reward giving too little insulin. We check both directions.
A change is blocked if it raises low blood sugar riskLow blood sugar riskWhat it is. How often and how far glucose falls below safe levels: below 70 mg/dL (level 1) and below 54 mg/dL (level 2, clinically serious).Why it matters. Severe lows can cause confusion, seizures, or worse. A change must never make them more likely. or high blood sugar riskHigh blood sugar riskWhat it is. How long glucose stays far above range (above 250 mg/dL), a warning sign for diabetic ketoacidosis.Why it matters. A gate that only checks lows would reward giving too little insulin. We check both directions.. Checking only one would reward the wrong behavior.
Same patients, same bad daysBase and headWhat it is. Base is the dosing code before the change; head is the code with the change. The trial runs both on the same patients, meals, faults, and random seed.Why it matters. Because everything else is identical, any difference in outcomes comes from the code change.
Old and new code face identical meals, sensor faults, pump records, and random seedsRandom seedWhat it is. A number that fixes the sensor noise and other randomness, so a run can be repeated exactly.Why it matters. The same inputs always give the same result, so a verdict can be reproduced and checked by anyone., so any difference comes from the code.
Small but consistent harm countsPaired consistency checkWhat it is. For each patient and scenario, compare base and head. If a risk index rises by 15% or more on average and rises in so many pairs that chance is ruled out (exact sign test, p at most 0.01), the change is blocked.Why it matters. A bug that makes things a little worse almost every time is real harm, even if the average change is small.
The gate also blocks a change that makes things a little worse almost every time, using a paired statistical test.
How it works
Full walkthroughA change is proposedMerge requestWhat it is. A proposed change to the code, reviewed before it is merged. GitLab's name for a pull request.Why it matters. Every dosing change arrives as one, so that is where the trial runs and where the verdict appears.
A developer opens a merge request on the dosing code.
An agent plans the trialGitLab Duo flowWhat it is. An AI agent workflow on GitLab's Duo Agent Platform. Ours start on their own: when a merge request opens, when its pipeline fails, or when an incident opens.Why it matters. Our three flows plan the trial, explain the verdict, and respond to incidents. They can read and comment, the planner can make the trial bigger, and the incident agent can undo a change with git revert. None can write dosing code.
A GitLab Duo flow reads the change, names its risks, and can make the trial bigger.
The trial runsScenariosWhat it is. Eight simulated days, each with meals and at most one fault: a normal day, a forgotten bolus, a large dinner, a sensor dropout, a false low, a sensor jump, a pump history that starts with a resume, and a bolus missing from history.Why it matters. Each fault is modeled on a failure reported in real dosing software or a recall, so the trial tests what has actually gone wrong before.
Old and new code dose the same virtual patients through eight realistic days, including sensor faults and missing pump records.
The gate decidesSafety gateWhat it is. The pipeline step that reads the trial's verdict and passes or blocks the change.Why it matters. It turns the trial into a decision: risky changes cannot merge or ship.
If low or high blood sugar risk rises past human-set limits, the change is blocked.
An agent explainsGitLab Duo flowWhat it is. An AI agent workflow on GitLab's Duo Agent Platform. Ours start on their own: when a merge request opens, when its pipeline fails, or when an incident opens.Why it matters. Our three flows plan the trial, explain the verdict, and respond to incidents. They can read and comment, the planner can make the trial bigger, and the incident agent can undo a change with git revert. None can write dosing code.
A second flow explains the verdict in plain words on the merge request, quoting the exact numbers.
Ship and watchCanary releaseWhat it is. A new engine version is deployed but receives no traffic until it passes a safety check on 151 dosing decisions. Then it gets all traffic and is checked again.Why it matters. A broken release never serves anyone. This caught a real crash on Oct 13: the bad version never received traffic.
A safe engine goes out as a canary, then ten virtual patients live on it around the clock.
The AI tests and explains. People decide.
Three GitLab Duo flowsGitLab Duo flowWhat it is. An AI agent workflow on GitLab's Duo Agent Platform. Ours start on their own: when a merge request opens, when its pipeline fails, or when an incident opens.Why it matters. Our three flows plan the trial, explain the verdict, and respond to incidents. They can read and comment, the planner can make the trial bigger, and the incident agent can undo a change with git revert. None can write dosing code. plan trials, explain verdicts, and respond to incidentsIncident response agentsWhat it is. When the virtual clinic's patients do worse on a new engine version, an hourly job opens a GitLab incident and calls two agents. One finds the dosing change behind it; the other undoes exactly that change in a new merge request.Why it matters. Fixing a bad release starts within the hour, with no one on call. The undo still goes through the full trial, and a person approves it by merging.Source. They can read and comment, the planner can make a trial bigger, and the incident agent can undo a shipped change with git revert. They can never write dosing code, the trial, or the gate limitsGate limitsWhat it is. The thresholds a change must stay within, in gate/thresholds.yaml. For example, time below 70 may rise by at most 0.5 percentage points.Why it matters. People own these numbers through code review (CODEOWNERS). Agents can never change them., a pipeline job enforces that (the agent boundaryAgent boundaryWhat it is. A pipeline job that fails if any AI agent account committed a change to dosing code, the trial, or the gate limits. The only exception is an exact undo (git revert) of a commit already on the branch, verified by comparing the two patches.Why it matters. The rule does not depend on what an agent's instructions say. It is enforced by the pipeline, and tested against four ways to abuse the exception.), and a person approves every merge.
What Virtual Trial will never doChecking...
Live right nowVirtual clinicWhat it is. Ten virtual patients living on the deployed engine. Every 10 minutes they move 2 simulated hours forward, and every insulin decision comes from the live engine.Why it matters. If a release made dosing worse, it would show up here, as worse outcomes for these patients.
Ten virtual patients live on the deployed engine. Every 10 minutes they move 2 simulated hours forward, and every insulin decision comes from the same engine the public can call.
Release safety, already provenCanary releaseWhat it is. A new engine version is deployed but receives no traffic until it passes a safety check on 151 dosing decisions. Then it gets all traffic and is checked again.Why it matters. A broken release never serves anyone. This caught a real crash on Oct 13: the bad version never received traffic.
On Oct 13 a new engine build crashed at startup. The canaryCanary releaseWhat it is. A new engine version is deployed but receives no traffic until it passes a safety check on 151 dosing decisions. Then it gets all traffic and is checked again.Why it matters. A broken release never serves anyone. This caught a real crash on Oct 13: the bad version never received traffic. never gave it traffic; the public engine kept serving the previous version, and the fix shipped the same hour.