How it works
From code change to safe release
Seven steps. A person starts it and people own every limit. Agents plan, explain, and respond to incidents; the pipeline tests, decides, and deploys; a person approves every merge.
A change is proposedMerge requestWhat it is. A proposed change to the code, reviewed before it is merged. GitLab's name for a pull request.Why it matters. Every dosing change arrives as one, so that is where the trial runs and where the verdict appears.
A developer opens a merge request that changes the dosing code. That human action starts everything.
Merge request on the GitLab project (oref0 imported with full history). The pipeline runs tests, security scans, the agent boundary check, the trial, and the gate.
An agent plans the trialGitLab Duo flowWhat it is. An AI agent workflow on GitLab's Duo Agent Platform. Ours start on their own: when a merge request opens, when its pipeline fails, or when an incident opens.Why it matters. Our three flows plan the trial, explain the verdict, and respond to incidents. They can read and comment, the planner can make the trial bigger, and the incident agent can undo a change with git revert. None can write dosing code.
When the merge request is opened, the planning agent starts on its own. It reads the change, names what could go wrong, and can make the trial bigger (all 30 patients). It cannot make it smaller or skip it.
GitLab Duo flow plan_trial (AI Catalog; merge request created trigger, or a mention). Tools: read the MR and diff, add the trial::full label, post a note. No tool can change code.
The trial runsScenariosWhat it is. Eight simulated days, each with meals and at most one fault: a normal day, a forgotten bolus, a large dinner, a sensor dropout, a false low, a sensor jump, a pump history that starts with a resume, and a bolus missing from history.Why it matters. Each fault is modeled on a failure reported in real dosing software or a recall, so the trial tests what has actually gone wrong before.
Old and new code each dose the same virtual patients through ten realistic days: a normal day, a forgotten bolus, a large dinner, a basal rate set too high, and five faults modeled on real bugs. Then a hunter searches dozens of varied days for the one that hurts the new code most.
trial/run.py: base (git worktree of the MR's base) and head run in parallel; each decision comes from oref0 via a persistent Node worker; 1-minute physiology (simglucose UVA/Padova), 5-minute loop, fixed seeds. trial/hunt.py: seeded search over meal times and sizes, mis-counted meals, sensor gaps, false lows, and basal errors, split across the same CI shards.
The gate decidesSafety gateWhat it is. The pipeline step that reads the trial's verdict and passes or blocks the change.Why it matters. It turns the trial into a decision: risky changes cannot merge or ship.
If low or high blood sugar risk rises past limits that people set, the change is blocked. It is also blocked if risk rises a little but consistently across almost every patient and day, or if the hunter finds a day where the new code causes a severe low the old code did not, and that holds up when the day is replayed with small changes. If the change also clearly lowers low risk overall, the gate does not decide alone: it asks a person to weigh the two.
Pooled limits plus a paired one-sided exact sign test on LBGI and HBGI (block if mean rise of 15% or more and p of 0.01 or less); a new severe low on the hunted day, confirmed on 12 variations of it (more severe lows, and lower glucose in a consistent majority by sign test). If paired LBGI also falls 15% or more consistently, the result is REVIEW (CI exit 3, a warning) and a person decides. Limits in gate/thresholds.yaml, owned through CODEOWNERS.
An agent explainsGitLab Duo flowWhat it is. An AI agent workflow on GitLab's Duo Agent Platform. Ours start on their own: when a merge request opens, when its pipeline fails, or when an incident opens.Why it matters. Our three flows plan the trial, explain the verdict, and respond to incidents. They can read and comment, the planner can make the trial bigger, and the incident agent can undo a change with git revert. None can write dosing code.
When a merge request pipeline fails, the explain agents start on their own. They read the gate's numbers and the code change and posts a plain-language explanation, quoting the numbers exactly.
GitLab Duo flow explain_verdict, two agents. An investigator reads the MR, diff, and job logs with read-only tools; a writer turns its findings into one note and never sees raw logs. Forbidden from recomputing metrics or suggesting a dosing patch.
Ship and watchCanary releaseWhat it is. A new engine version is deployed but receives no traffic until it passes a safety check on 151 dosing decisions. Then it gets all traffic and is checked again.Why it matters. A broken release never serves anyone. This caught a real crash on Oct 13: the bad version never received traffic.
A merged change deploys the dosing engine as a canary: no traffic until 151 test decisions stay inside oref0's safety limits. Then ten virtual patients live on it, around the clock.
Keyless deploy (Workload Identity Federation) to Cloud Run with --no-traffic and a tag URL; engine/smoke.js gates promotion; automatic rollback on failure. Virtual clinic: Cloud Run job every 10 minutes.
Agents respond to an incidentIncident response agentsWhat it is. When the virtual clinic's patients do worse on a new engine version, an hourly job opens a GitLab incident and calls two agents. One finds the dosing change behind it; the other undoes exactly that change in a new merge request.Why it matters. Fixing a bad release starts within the hour, with no one on call. The undo still goes through the full trial, and a person approves it by merging.Source
If the virtual patients do worse on the new engine, an incident opens on its own. Agents find the dosing change behind it and open a merge request that undoes exactly that change. The trial checks the undo, and a person approves it by merging.
Hourly scheduled pipeline: ci/clinic_watch.py opens the incident and mentions the respond_to_incident flow. Investigator (read only) names the suspect commit; responder runs git revert on a new branch and opens an MR with trial::full. ci/agent_boundary.py accepts an agent commit under lib/ only if it is the exact inverse of a commit already on the target branch (same patch-id).
What one trial measuresTime in rangeWhat it is. The share of the day glucose stays between 70 and 180 mg/dL, the target range in the international consensus.Why it matters. It is the headline measure of how well a person's glucose is controlled.Source
For every virtual patient and every scenario, under both versions of the code, on true blood glucose:
- Time in rangeTime in rangeWhat it is. The share of the day glucose stays between 70 and 180 mg/dL, the target range in the international consensus.Why it matters. It is the headline measure of how well a person's glucose is controlled.Source, 70 to 180 mg/dL
- Time below 70, and below 54 (serious lows)Low blood sugar riskWhat it is. How often and how far glucose falls below safe levels: below 70 mg/dL (level 1) and below 54 mg/dL (level 2, clinically serious).Why it matters. Severe lows can cause confusion, seizures, or worse. A change must never make them more likely.
- Severe low eventsSevere low eventWhat it is. Glucose below 54 mg/dL for at least 15 minutes in a row.Why it matters. This is the consensus definition of a clinically significant low. Any new event blocks a change., below 54 for 15 minutes or more
- Time above 250, a warning sign for ketoacidosis High blood sugar riskWhat it is. How long glucose stays far above range (above 250 mg/dL), a warning sign for diabetic ketoacidosis.Why it matters. A gate that only checks lows would reward giving too little insulin. We check both directions.
- Risk indexes for lows and highs, LBGI and HBGILBGI and HBGIWhat it is. Low and High Blood Glucose Indexes (Kovatchev): single numbers that weigh how far and how often glucose goes low or high, with larger excursions counting more.Why it matters. They capture risk that a simple percentage of time misses, such as one deep low.
The faults we injectFault injectionWhat it is. Deliberately adding real-life problems to the trial: sensor dropouts, false readings, missing pump records, forgotten boluses.Why it matters. Many real dosing bugs only appear when data is messy, not on a perfect day.
Real dosing bugs often hide in messy data. Each fault is modeled on a reported failure:
- A 45-minute sensorCGMWhat it is. A continuous glucose monitor: a small sensor under the skin that reports glucose every few minutes.Why it matters. It is the algorithm's only view of the body. Gaps and false readings are realistic faults. dropout after dinner (Tandem Class I recall, Trio #989)
- A false low overnight from pressing on the sensor
- A sensor that returns after a gap with a 60 mg/dL jump (Trio #989)
- Pump history that starts with a resume (Trio #898)
- A dinner bolusBolusWhat it is. A one-off dose of insulin, usually at a meal.Why it matters. Forgetting one, or the algorithm not knowing about one, are common real-world problems the trial tests. missing from history while it is delivered (AndroidAPS #2580, Loop #632)
- A forgotten lunch bolus, and a dinner almost twice the usual size
Where it runs
GitLabPipelineWhat it is. The automatic sequence of jobs GitLab runs for each change: tests, security scans, the trial, the gate, and deployment.Why it matters. It makes the safety check impossible to skip: a blocked gate stops the change.
Merge requests, two Duo agent flows and a chat agent, CI/CD pipelines, security scans, the agent boundary check, and the gate.
Google CloudGoogle Cloud RunWhat it is. Google's service for running containers on demand. Our dosing engine, website, and virtual clinic run on it.Why it matters. It scales to zero when idle, so the live demo costs almost nothing.
The dosing engine, website, and virtual clinic on Cloud Run, deployed with no stored keysKeyless deploysWhat it is. GitLab's pipeline proves its identity to Google Cloud with a short-lived token (Workload Identity Federation), so no password or key is stored anywhere.Why it matters. No secret to leak. Google only trusts this project, on protected branches., monitored with uptime checks and alerts. All defined in Terraform.
Open sourceOpenAPS oref0What it is. The open-source algorithm that decides, every 5 minutes, how much insulin a pump should give, based on glucose readings and insulin already given. MIT licensed.Why it matters. It is the project Virtual Trial protects. Its logic was carried into other open-source dosing apps (AndroidAPS, Trio).Source
OpenAPS oref0 (MIT) is the dosing code under test. simglucose (MIT) provides the UVA/PadovaUVA/Padova simulatorWhat it is. A mathematical model of how glucose and insulin move through the body, developed at the University of Virginia and the University of Padova.Why it matters. The FDA accepted the commercial version in 2008 as a substitute for animal trials in testing insulin controllers. We use an open-source implementation and do not claim FDA validation.Source virtual patients.