Compiling Medical Evidence: A Clinical Research Compiler That Turns Guidelines and Trials Into a Queryable Knowledge Artifact

Compiling Medical Evidence: A Clinical Research Compiler That Turns Guidelines and Trials Into a Queryable Knowledge Artifact

By Daniel Kliewer · 2026-07-21

This is a working build of the Clinical Research Compiler: the Research Compiler Agent SDK evolved into a medicine-specific tool. It compiles real guidelines, trials, and reviews into one provenance-tracked Clinical Knowledge Artifact, and renders it as a live explorer at clinical-evidence-explorer.

1. The shift: from "retrieve papers" to "compile evidence"

Traditional medical AI looks like this:

Question → Retrieve Papers → Generate Answer

That works once. The next question re-reads the web from scratch, re-embeds the same PDFs, and re-derives what was already known — and nothing is auditable, because the "answer" has no citeable substrate.

The Research Compiler flips the shape:

Medical Literature → Knowledge Compilation → Evidence Graph → Research Agent → New Discoveries → Updated Knowledge

The goal is not to replace doctors or researchers. It is to reduce the cost of understanding the knowledge they need — and to make that understanding auditable: every fact traces to a source, and every source carries an authority score. This is the same "Compile-Time Intelligence" move I wrote about for general research, specialized for the clinical domain.

2. Why clinical evidence is the right first vertical

The clinical literature is a near-perfect target for a compiler:

• It is structured by convention — guidelines, trials, systematic reviews, case reports — so extraction is tractable.
• It already speaks in grades and strengths (GRADE, strength-of-evidence), which a compiler can make explicit and queryable.
• Conflicts are the product. HbA1c targets differ between NICE, WHO, ADA, and CDC. First-line therapy recommendations shift as SGLT2 inhibitors and GLP-1 agonists accumulate outcome data. A compiler that surfaces those conflicts with provenance is genuinely useful.
• It rewards provenance more than any other domain — a claim without a citable guideline is clinically worthless.

So we picked a solvable, high-value, well-scoped problem: Type 2 diabetes management — compile the guidelines and literature into a graph of treatments, evidence grades, treatment conflicts, and guideline evolution.

3. Two repos, one pipeline

The deliverable is split the way a real toolchain should be:

• Repo A — clinical-research-compiler-sdk (github): the evolved SDK. It keeps the v1 engine (pass registry, orchestrator, content-hashed artifact store, local-first inference client) and adds a heavily customized clinical layer.
• Repo B — clinical-evidence-explorer (github, live): the generated Next.js app that renders the compiled artifact. It is rebuilt from Repo A's explorer generator on every compile. The app is a window; the artifact is the product.

This separation matters: the intelligence lives in the compiled clinical-knowledge-artifact.json, not in the UI. You can re-skin, re-deploy, or query the artifact without touching the compiler.

4. What the clinical layer adds over v1

Every clinical pass is declared, not hardcoded. Drop a directory with a pass.yaml, and the existing PassRegistry discovers it and the Orchestrator plans the dependency graph around it. Adding another clinical dimension — biomarker↔outcome links, rare-disease case clustering — is one more directory, zero core edits.

The evidence grade uses a transparent, decomposable rubric rather than a black-box score:

grade = f(study_design, consistency, population_size, guideline_alignment, safety_margin)

so a grade of A means "multiple guideline bodies + RCT support," not "the model felt confident."

5. The run (copy-paste)

Local-first. Inference runs on Ollama (llama3.1:8b for chat, nomic-embed-text for embeddings) at localhost:11434. No cloud API, no key.

# 0. dodge a shell PYTHONPATH leak (agent/host envs preload a venv)
export SDK=~/Projects/clinical-research-compiler-sdk
cd "$SDK"

# 1. venv + pinned deps (jsonschema==4.17.3 avoids the referencing/rpds conflict)
/opt/homebrew/bin/python3.12 -m venv .venv
env -u PYTHONPATH .venv/bin/python -m pip install -r requirements.txt

# 2. DISCOVER real clinical sources (--clinical = medical authority overlay)
env -u PYTHONPATH .venv/bin/python researchc.py discover build-t2d \
  --objective "Type 2 diabetes management" \
  --seeds "https://www.nice.org.uk/guidance/ng28" \
  --clinical --max 10

# 3. COMPILE to the clinical artifact (Ollama model passes)
env -u PYTHONPATH .venv/bin/python researchc.py compile build-t2d --local

# 4. EXPLORE -> generates the Next.js app
env -u PYTHONPATH .venv/bin/python researchc.py explore build-t2d
# -> build-t2d/clinical-evidence-explorer/

# 5. sanity-check
env -u PYTHONPATH .venv/bin/python -c \
  "import json;d=json.load(open('build-t2d/clinical-knowledge-artifact.json'));print(d['clinical_summary'])"

6. What the real run produced

Objective: Type 2 diabetes management — 10 live sources (NICE, NIDDK, ADA, AACE, PMC, EMJ, Wikipedia) discovered with the clinical authority overlay, compiled on llama3.1:8b:

• 6 treatments extracted and stratified: Metformin (first-line, grade A), SGLT2 inhibitors (grade A), GLP-1 receptor agonists (grade A), sulfonylureas / DPP-4 inhibitors / TZDs (grade B).
• 5 treatment conflicts, including a high-severity head-to-head: SGLT2 inhibitors vs GLP-1 receptor agonists are both recommended first-line — the kind of tension a clinician actually needs surfaced.
• Guideline evolution: the HbA1c target shifted (<7% → less than 7%), and SGLT2 inhibitors / Metformin were promoted to first-line in recent editions.
• Corpus confidence 0.581 — a number, not a vibe, computed from authority × corroboration × recency × (1 − contradiction-rate).

That is the whole point: the compiler did not "summarize papers." It built a structured, graded, conflict-aware evidence graph that a human can audit and query — and it is reproducible. Re-running with the same URLs and model reuses every unchanged artifact.

7. Deploying the explorer to Vercel (Repo B)

# clone the empty Vercel repo and drop the generated app in
git clone https://github.com/kliewerdaniel/clinical-evidence-explorer
cp -R build-t2d/clinical-evidence-explorer/* clinical-evidence-explorer/
cd clinical-evidence-explorer
git add -A && git commit -m "explorer" && git push -u origin main

# deploy (the app's scripts/copy-artifact.mjs copies the artifact into data/ at build time)
vercel deploy --prod --project clinical-evidence-explorer --archive tgz

The explorer embeds clinical-knowledge-artifact.json (no server, no API at runtime) — it is a static renderer of a compiled artifact.

Build gotcha worth recording: Next 14 / SWC rejects return <div> on a single line and chokes on a block-element close immediately followed by a map/arrow )) (</div>))). The generator writes attributes with single quotes and splits the )) onto its own line; the build transform (_jsx_attr_to_double_quotes + _split_block_close_parens) fixes both. framer-motion's <motion.div> as a block element also fails this build config, so the explorer drops it (animations are cosmetic).

8. Reproducibility is the product

• Every artifact is content-hash cached; compile --incremental reuses unchanged work.
• Every fact carries a source_id that joins back to the provenance table (URL, author, fetch timestamp, content hash).
• The citation-quality pass turns that into a measurable corpus confidence. The compiled clinical-knowledge-artifact.json is the auditable substrate. The explorer is just a renderer.

8b. Iteration 2: from 58% to 76% confidence, and a real UI

The first compile landed at 0.58 corpus confidence — not because the evidence was weak, but because two design choices in the confidence pass were over-punishing, plus one model-output bug:

1. Corroboration was exact-string only. Differently-phrased recommendations from different guidelines (NICE "Metformin is first-line" vs ADA "Metformin is the main first-line medication") never matched, so corroboration sat at 0.00 on 8 of 10 sources — with a 0.35 weight, that alone capped confidence.
2. Contradiction was scored at the source level. A source with a single claim was marked 100% "contradicted" if that one claim's text matched either side of any contradiction pair.
3. pass-r2 emitted false contradictions — unrelated sentence pairs ("Type 2 diabetes is chronic…" vs "Metformin is a biguanide…") that poisoned the contradiction term.

The fix was a legitimate SDK iteration, not a fudge:

• Paraphrase-aware corroboration (token-Jaccard ≥ 0.5) so re-worded recommendations from different guidelines corroborate.
• Claim-level contradiction attribution — only a source's claims that are actually in a real contradiction count, not all-or-nothing.
• Treatment-agreement corroboration — a source that backs a treatment other guideline sources also back (Metformin first-line in NICE + ADA + AACE) earns high corroboration. This is the honest signal a richer corpus was meant to surface.
• A same-subject filter on pass-r2 dropped the false positives, and a tightened pass-c4 prompt stopped co-recommended treatments being reported as conflicts (65 false positives → 7 real ones).

We also enriched the corpus to 16 high-authority sources (added WHO, AHA-professional, Medscape, BMJ Best Practice, EMJ ×2, Ovid, ADA Standards) so recommendations genuinely converge. Result: 0.58 → 0.76, all 16 sources at 0.00 contradiction rate, with 13 treatments and 7 real treatment conflicts.

The UI: Meta's Astryx design system

The original explorer was hand-rolled Tailwind. For the upgrade I adopted facebook/astryx — Meta's open-source design system (150+ accessible components, brand-level theming, dark mode, MIT). Crucially, Astryx ships pre-built CSS and typed React components, so there is no StyleX build plugin to wire up: the app imports three CSS files and wraps the tree in <Theme> + <LinkProvider>.

Because Astryx requires React 19, I bumped the explorer from Next 14 / React 18 to Next 15 / React 19 (the clean path its own example uses). The generator (pass-r10-explore-clinical) now emits the Astryx app directly — SideNav, Card, Badge (with semantic success/info/warning/error variants for grades and severity), Button grade filters, TextInput search, Heading/Text/Grid. The compile loop now yields a polished, accessible UI automatically, keeping the "the loop is the product" architecture. The deployed explorer is live at clinical-evidence-explorer.vercel.app.

Build gotcha (Iteration 2): @astryxdesign/core peer-requires react >= 19 and @stylexjs/stylex. Stay on Next 15 / React 19 — a --legacy-peer-deps force-install onto React 18 is fragile. Astryx's Badge takes a label prop (not children) and Card has no CardHeader subcomponent (use <Heading> inside), and TextInput requires a label prop; these are the three import-API details that will break a blind port.

9. The pattern, restated

Compile-Time Intelligence — move semantic computation from runtime inference into reusable, queryable knowledge artifacts.

We stopped re-parsing source code on every program run. We should stop re-reading the medical literature on every clinical question. Compile it once. Grade it. Surface the conflicts. Recompile only when the guideline frontier moves.

The Clinical Research Compiler is a working instance of that pattern for medicine — local-first, provenance-tracked, reproducible, and deployed.

10. Try it / replicate it

• SDK (Repo A): github.com/kliewerdaniel/clinical-research-compiler-sdk
• Live explorer (Repo B): clinical-evidence-explorer.vercel.app
• The SDK ships a bundled skills/clinical-research-compiler/SKILL.md that reproduces this entire run for a new agent or human.

This project extends the Research Compiler Agent SDK (v1) and the Knowledge Compiler SDK, and builds on the "intelligence is the accumulated decisions" philosophy from Sovereign AI.