In our earlier essays we argued that an AI agent should compose tested analysis nodes rather than rewrite code from scratch (first essay), that the rules for auditing a study should run as software while the work is being done (second essay), and that researchers and agents need an inspectable, shared environment beyond the chat window (third essay). In our fourth essay we looked inside a single study and asked why five axiom systems from reverse mathematics [1], which we read as five moves—Execution (), Choice (), Stabilization (), Stratification () and Self-Reflection ()—govern how our seven skills audit a manuscript, a figure or a statistical table.
Yet no scientist designs a two-year project or writes a grant from a single paper. Before ordering a vial of antibody or choosing a gene target, a researcher has to read across a whole topic: dozens or hundreds of papers published over decades by different laboratories, using different assays, cohorts and statistical conventions. How can these single-paper checks help a scientist make sense of an entire field—and keep that answer current as new studies appear every week?
The answer continues the story we told in the fourth essay: how human knowledge climbs past its working-memory limit by packing proven chains into building blocks, and then running the same five cognitive moves one level higher across those blocks.
Why literature reviews and AI summaries suffer from lossy packing
As we saw in the fourth essay, human working memory holds about four chunks at once, and by our loose analogy a chain of four quantifier flips () is where even practised mathematicians struggle. Mathematicians climbed past that ceiling by conceptual compression: packing a multi-step quantifier chain into a named black box—such as uniform convergence—so that the next theorem can treat the whole chain as a single noun. The compression costs the reader something, because a name hides the dependencies underneath it. Yet it can always be undone, because the definition keeps every detail.
Empirical science relies on the exact same trick, and calls it a citation. When Paper B cites Paper A in its Introduction—"Pathway promotes cartilage degradation in osteoarthritis [A]"—it packs a thirty-page experimental report into a one-sentence claim so the field can move on to the next question.
The trouble is that a citation is compression that cannot be undone. Behind a mathematical name stands a definition that anyone can unfold. Behind a citing sentence stands only a pointer to the paper: recovering the conditions means reopening that paper and auditing it again, which the next author rarely does. In practice the one-sentence summary travels on without almost every condition that determined whether Paper A's claim actually held:
- Did the reported -values in Paper A match its own test statistics and degrees of freedom? Nuijten and colleagues ran the
statcheckR package across 30,717 articles in eight psychology journals and found checkable APA-formatted null-hypothesis tests in 16,695 of them (258,105 -values in total). Half of those articles held at least one -value inconsistent with its own test statistic and degrees of freedom: 49.6% of the 16,695. In 2,150 articles (12.9%), recalculating the -value changed whether the result was significant at the threshold [2]. - Did the biological assay in Paper A actually measure its named target? In October 2026 Sholto David reported a search of 20 commonly used cell lines that carry confirmed genetic deletions or biallelic inactivating mutations, covering 10 proteins. He found 230 Western blot reports, in 221 published papers, that claimed to detect the missing protein without the gene having been re-expressed. Examples are caspase-3 in MCF-7 cells (49 papers), BAX in Jurkat cells (47 papers) and STAT3 in PC-3 cells (35 papers) [3].
- Was Paper A only run in male mice, at a single time point, or on mRNA where the paired protein assay failed to change?
- Was Paper A even still valid when Paper B cited it? When Hsiao and Schneider examined 7,813 retracted PubMed papers and 48,134 in-text citation contexts in the PubMed Central open-access corpus, retraction did not change how the retracted papers were cited, and among 13,252 post-retraction citation contexts only 5.4% (722) acknowledged the retraction [4].
Each row is a different study with its own denominator, so the rows are not added together. The hollow marker is a measured zero. Sources: references [4], [10] and [11].
The loss can be traced. Steven Greenberg followed one belief, that the protein β-amyloid is abnormally and specifically present in the muscle fibres of patients with inclusion body myositis, through 242 papers and 675 citations. Ten of those papers had tested the claim with primary data: four supported it, and six weakened or refuted it. Of the 214 citations to those ten papers, 94% went to the four supportive ones. He also documented a hypothesis that became a fact through citation alone [19].
The paper itself keeps less than we assume. When the Reproducibility Project: Cancer Biology set out to repeat 193 experiments from 53 high-impact papers, not one could be designed from the published paper and its supplements alone; every one needed clarifications [20]. Reopening a paper recovers only what was written down; where a protocol detail is missing, an audit has to record the gap, not assume the condition was met.
When a human review article—or a standard retrieval-augmented AI tool that summarises abstracts—packs fifty papers by reading only their headline conclusions, it zips those broken assumptions, selective citations and unstated conditions into an apparent consensus. As Derek Lowe warned in Science when commenting on David's missing-protein preprint, the AI systems that we are told will read the whole literature for scientists will be fed those erroneous blots alongside everything else [5]. Even formal systematic reviews and meta-analyses fall into the same trap when they pool primary studies without auditing them first, and there are now a great many of them. John Ioannidis examined the growth of PubMed-indexed syntheses between 1991 and 2014. Annual publications rose 2,728% for systematic reviews and 2,635% for meta-analyses, against 153% for PubMed overall. Antidepressants for depression alone received 185 separate meta-analyses between 2007 and 2014. In genetic epidemiology, he judges the flood of candidate-gene meta-analyses almost all misleading, because they pooled fragmented and selectively reported studies [6].
Packing that keeps the conditions: turning a screened paper into an audited Evidence Capsule
To stop errors from compounding when we scale from one manuscript to a whole topic, packing must be audited before it is compressed. This is the design we are building. Instead of storing a paper in our database as a flat text summary or an embedding vector of its abstract, we use our single-paper auditing workflows as the compiler that packs each paper into a structured Evidence Capsule. Those workflows are the biomedical-literature-evidence skill (validation record), the academic-peer-review skill (skill page, benchmark, PeerJ 20995 blind review), the scientific-figure-master skill, the scientific-writing skill (release notes), the meta-analysis skill (performance report) and the Mendelian randomization skill (reproduction report). It follows the same idea as Big5c, the Big-5 compiler of the fourth essay: organize the review on the five rungs. Big5c applies that review to an AI agent's proposed action; the capsule compiler applies it to a paper.
An Evidence Capsule is a scientific zip file that keeps the conditions. Alongside the paper's bibliographic identity and extracted claims, it preserves the five Big Five verification coordinates of the study:
- Rung 1 ( Execution receipt): Recomputed test statistics (
t,F,z, , odds ratios and standard errors), exact sample sizes, and parsed physical units (ug/gkept strictly apart fromparticles/g). - Rung 2 ( Experimental choice tree): The exact experimental branch the claim rests on—species, strain, sex, age or exposure duration, tissue compartment, mRNA versus protein readout, and the cell line and antibody checked against public databases such as Cellosaurus, the lookup we described in the second essay.
- Rung 3 ( Multiplicity receipt): How many hypotheses the paper tested in the same family, and whether the headline result survived family-wise or false-discovery-rate correction.
- Rung 4 ( Stratum and provenance tag): Which document section the statement came from (
MethodsandResultsseparated from literature remarks inIntroduction,DiscussionandReferences), which underlying patient cohort or dataset ID was analysed, and where the study design sits on the evidence hierarchy. - Rung 5 ( Confounder and status ledger): Which confounders, negative controls or pleiotropy checks were actually run, which remain
unavailable, and whether the paper or any of its key references carries a correction, expression of concern or retraction notice.
Narrow AI nodes—each with a customised prompt, skill and context loading, and a gated hook for quality control—extract the study design tree and candidate claims into explicit schemas; deterministic code nodes recompute the arithmetic, check the cell lines and units, and lock the capsule's verification state.
A design diagram. Each capsule field answers one rung of the ladder for a single paper; the same rung applied across all capsules gives the field-level move on the right.
The second-order Big Five ladder: automated meta-science across a field
In the story we told in the fourth essay, once a complex process is packed into a verified building block, mathematicians do not invent a sixth way of thinking: they step onto the new foundation and run the same Big Five ladder () one level higher.
Packing published papers into audited Evidence Capsules lets Nexitia and Nexitia Core (listed today as AROS Core) do the same thing for a scientist's research topic. When a researcher opens a topic of interest, the system climbs a second-order Big Five ladder across all capsules in that field:
1. Field-level (Execution): Cohort deduplication and unit harmonisation
Before two studies can be compared or pooled, deterministic code nodes check the extracted cohort identifiers, recruitment windows and physical units to test whether the studies are independent and commensurable. Using the study-and-report mapping of the biomedical-literature-evidence skill, the engine links the reports of one study to that study. It records separately where two studies may share participants, as papers from one hospital registry or biobank often do, and shows an unresolved overlap instead of counting it twice. It aligns effect directions and transforms compatible scales while refusing to pool incommensurable physical units without a verified conversion factor.
2. Field-level (Choice): Cross-checking conclusions and "unzipping" contradictions
In almost every active field, a scientist eventually meets a contradiction: Laboratory A reports that a target is protective, while Laboratory B reports that it is harmful or null. A conventional review smooths the conflict into a polite sentence ("results remain controversial"). The engine first checks that the two results address the same question and really disagree, since a significant estimate and an inconclusive one can be compatible.
Because every Evidence Capsule preserves its experimental choice tree, our field-level engine maps the capsules onto a shared branching graph and unzips conflicting pairs side by side on the Nexitia canvas. Instead of a vague controversy, the scientist sees the exact branch where the two laboratories diverged:
- Did Laboratory A measure mRNA at 6 weeks in male mice while Laboratory B measured protein at 24 weeks in female mice?
- Did one study use PC-3 cells (which lack endogenous STAT3 [3]) while the other used primary human tissue?
- Did one epidemiological study assess a 4.6-year randomized drug exposure while another assessed a 54.9-year lifelong genetic exposure [7]?
Unzipping the contradiction turns a deadlock in the literature into a short list of experiments that can tell the competing explanations apart.
3. Field-level (Stabilization): "Audit-before-pool" living synthesis
When multiple capsules measure the same quantitative contrast, the meta-analysis skill—whose RevMan-compatible engine reproduced 1,060 of 1,061 pooled analyses and 10,797 of 11,351 checkable quantitative items (95%) across 28 Cochrane reviews—computes the pooled effect, heterogeneity (), prediction intervals and small-study bias flags.
Crucially, because every capsule carries its single-paper audit receipt, the database computes two syntheses side by side:
- The published pool: the estimate before the capsule checks are applied, with every published result taken as reported.
- The Big-Five audited pool: what survives after removing retracted studies, deduplicating overlapping cohorts, flagging results whose recalculated -value changes their significance (the gross inconsistencies that
statcheckdetects [2]) for a person to check, showing the pooled result with and without those that stay unresolved, and separating incompatible assays.
A forty-paper "consensus" can collapse to two small, inconsistent human cohorts after audit-before-pool filtering; seeing that early can save a laboratory eighteen months and a grant budget spent chasing an artefact.
4. Field-level (Stratification): Evidence grading that keeps hierarchies apart
Not all capsules sit on the same rung of reality. Applying the principle behind the GRADE framework—which rates evidence quality from study design downward for study limitations, inconsistency, indirectness, imprecision and reporting bias, and upward for large effects or dose–response gradients [8]—our stratification engine places every capsule on a strict partial order:
This is a different order from the one in the fourth essay, which recorded how far each level stands from the claim being made. This one is our own ordering for a causal claim in humans. It borrows from GRADE the idea of starting from the study design and then rating the evidence down for its flaws or up for its strengths [8].
Because the strata are kept strictly apart, thirty cell-line papers using unvalidated antibodies can never outvote a well-powered human cohort or a properly instrumented Mendelian randomization study. Each topic question receives an explicit, inspectable evidence grade, anchored to the highest stratum that passes its audit checks and adjusted for the limitations that remain.
5. Field-level (Self-Reflection): Evidence triangulation and white-space discovery
In frontier biology and aetiological epidemiology, researchers rarely have twenty identical randomized trials to pool. More often they have heterogeneous lines of evidence: a prospective cohort, a Mendelian randomization analysis, a sibling comparison, an animal perturbation and a negative-control study.
As Lawlor, Tilling and Davey Smith formulated in their framework for evidence triangulation, causal confidence is strongest when results agree across multiple approaches whose key sources of bias are unrelated to one another (and ideally bias the estimate in opposite directions)—for example, pairing multivariable cohort regression (vulnerable to residual socioeconomic confounding) with two-sample Mendelian randomization (vulnerable mainly to horizontal pleiotropy rather than to lifestyle confounding) and randomized trial intermediates, while accounting for exposure duration and timing [7]. Conversely, when Lawlor and colleagues triangulated breastfeeding and later body mass index across cohorts, cross-context comparisons in low- and middle-income countries, a randomized promotion trial and negative-control outcomes (parental reports of home invasion by mice or pigeons), the negative controls and orthogonal designs supported residual confounding by socioeconomic and family background as the explanation for the apparent protective cohort association [7].
At Rung 5 (), our field-level engine operationalises triangulation across our skills: it compares the causal contrasts, exposure timing and expected biases of the different study designs and checks whether their results support one compatible explanation, flags which confounders or pleiotropy pathways remain untested across the entire topic, and highlights candidate scientific white spaces—a missing experiment or an unadjusted confounder—on the Nexitia canvas, ready to feed directly into the grant-write-master skill (validation record, fact-preservation record) or a CohortDAG (release notes) and Tracyte analysis workflow.
A living database with compiler-style dependency invalidation
A static literature database begins decaying the day it is compiled. Shojania and colleagues followed 100 quantitative systematic reviews of randomized trials indexed in ACP Journal Club. For 57% of them, a quantitative or qualitative signal appeared that called for an update, and the median review survived 5.5 years before that happened. Some aged much faster: 23% were out of date within 2 years of publication, 15% within 1 year, and 7% on the day they were published [9].
Worse, when an included trial is retracted later, static reviews almost never correct themselves. Kataoka and colleagues examined 587 systematic reviews and clinical practice guidelines that cited retracted randomized controlled trials. Of these, 127 had cited an already-retracted trial without caution, and none of them had corrected themselves by May 2022. Another 239 were published before the trial was retracted; among them, only 5% of systematic reviews (9 of 196) and 5% of guidelines (2 of 43) had corrected or retracted their results [10].
Many of these citations could have been avoided. Fanelli, Wong and Moher examined 229 meta-analyses published between 2013 and 2016 that cited a retracted study. For 156 of them (68%), the retraction had come at least a year before the citing meta-analysis was published, yet only 21 of those 156 avoidable citations (13%) excluded the study because it was retracted [11].
And as Bakker, Schneider and colleagues note in BMJ Evidence-Based Medicine, while removing a single retracted paper leaves the results of most meta-analyses largely intact, flawed or fabricated data can change conclusions: excluding two flawed studies invalidated a published review's finding that ivermectin lowered COVID-19 mortality, and excluding a single trial affected by data fabrication would alter more than half of the analyses in 22 meta-analyses of apixaban [12].
To solve this, Elliott, Higgins and colleagues proposed living systematic reviews: continually updated online syntheses where literature searches feed continuously into quality assessment, data extraction and meta-analysis, and they discuss sequential and Bayesian methods for handling the false positives that repeated updates can produce [13].
Our database architecture turns the living review into a compiled dependency graph built around the topics scientists subscribe to:
- Continuous ingestion and packing: For each topic a researcher or laboratory watches on Nexitia, the system monitors newly published articles and preprints across PubMed, Europe PMC and Crossref, compiles each new study whose full text it can read into an audited Evidence Capsule, holds abstract-only records as unverified, and attaches both to the topic graph.
- Compiler-style dependency invalidation: Because every pooled estimate, evidence grade and triangulation arrow in the database records its explicit dependency paths (
bound_paths) to the underlying Evidence Capsules, the database behaves like a software build system. If an upstream paper is retracted or corrected on Crossref/Retraction Watch, if a new audit check flags a reagent or statistical defect, every downstream synthesis node that depends on that capsule is automatically marked stale. A newly published study that enters a topic's set of eligible studies does the same to that topic's syntheses. Reproducible calculations then rerun, and decisions that need judgment return to a person. - Signal-gated human notification: Rather than flooding a researcher with daily paper alerts, the living ledger notifies the human scientist on the canvas (including downstream visual tools like AXIO) when a newly packed study or an upstream invalidation crosses a prespecified updating boundary, materially changes a pooled estimate or its uncertainty, changes an evidence grade, or resolves an unzipped contradiction. And through the shared-brain federation and MCP server described in our third essay (white paper, privacy policy, curated exchange), a verified capsule graph curated by one laboratory can be shared as a reusable scientific asset so the next researcher never has to re-screen the same three hundred papers from scratch.
Why the scientist stays at the centre
That architecture also clarifies how our approach differs from generic "AI scientist" pipelines that have appeared over the past two years:
- Auditing raw artefacts before pooling, rather than summarising abstracts or triples: Retrieval-augmented literature tools that summarise abstracts, flag sentence-level contradictions in text, or reduce papers into flat
(Subject, Predicate, Object)knowledge-graph triples carry forward whatever errors the published prose contains. An Evidence Capsule audits the published tables, the.xlsxsupplements and.docxchart caches that are available, multiplicity bounds, cell-line genetics [3],statcheckarithmetic [2] and retraction notices [4] before a single number enters the pool, and it records which checks lacked the inputs they need. - Executing verified workflow nodes, rather than improvising fresh scripts for every paper: End-to-end paper generators that ask a language model to write new analysis code from scratch can produce a manuscript that looks convincing to a human reader while hiding data-leakage bugs and figure-to-text contradictions. Yamada and colleagues submitted three autonomous manuscripts from The AI Scientist-v2 to peer review at an ICLR 2025 workshop. One of the three passed, with a score of 6.33 that placed it in the top 45% of submissions; the other two were rejected. Worse, the authors then audited the accepted paper. Re-running its synthetic data generator showed that about 57% of test items also appeared in the training set. The caption of Figure 3 misstated the validation-loss trend, and the prose called two architectures equal while Figure 5 showed one ahead [14]. In our architecture (first essay, second essay), agents compose deterministic code nodes and AI agent workflows that audit research rather than improvising uninspected scripts.
- Pairing deterministic software with human domain review on a GUI canvas, rather than having models grade models: Internal debate loops where one language model judges another model's ideas cannot substitute for empirical verification or domain expertise. In a blinded study where 79 NLP researchers scored research ideas, every language-model evaluator tested agreed with the expert rankings less often (43.3% to 53.3%) than the experts agreed with one another (56.1%) [15]. In Nexitia, software enforces the deterministic arithmetic and database rules, while a human scientist reviews the agent's choices on the tools' canvas using a human-friendly GUI.
Why keep the human scientist at the centre of an automated synthesis engine? Because physical biology and clinical medicine run on a clock that raw compute cannot compress. Using 41,040 industry drug-development paths (15,102 drugs, each counted once for every disease it was tested in), Wong, Siah and Lo estimated the probability of approval from Phase 1 at 13.8% [16], and each path takes years of wet-lab assays and clinical follow-up. Because pure in silico exploration across existing empirical datasets—human genetic instruments in the Mendelian randomization skill, trial tables in the meta-analysis skill, clinical cohorts in CohortDAG and single-cell atlases in Tracyte—runs in hours or days rather than months at the wet-lab bench, we expect a researcher to be able to test something like ten dry-lab hypotheses in a two-month cycle and to reserve bench experiments for signals that survive all five rungs of the audit.
Faster dry-lab exploration moves the bottleneck from generating candidates to verifying and digesting them. As Terence Tao observed in the written version of his 2026 International Congress of Mathematicians lecture, moving from an era of scarcity to an era of AI abundance exposes five stages behind solving a problem—generation, verification, exposition, community acceptance (digestion) and canonicalization into definitive theory—producing what he calls "proof indigestion" whenever raw machine solutions accumulate faster than human experts can verify, explain and digest them [17]. OpenAI's October 2026 bulk release of 722 solved mathematical problems, in more than 350 findings, shows the problem: among about ten of the papers that Tao read, one or two contained genuinely clever new ideas and one or two had already been withdrawn for errors [18]. Tao proposes giving priority to the first authors who present a result in a clear, public scientific exposition, making credit conditional on developing a proof to at least the publication stage, and shifting prestige toward those who verify and digest results [18]. In empirical biology as in mathematics, as AI systems take over exhaustive calculation and literature screening, a primary responsibility of the human scientist is initiating the right questions, auditing the AI's assumptions, and translating dense quantitative outputs into clear biological explanations that peers can understand and test. We are developing this human–AI verification workflow together with our research collaborators, whose participation directly builds both the engine and the living database through three inputs:
- Interested topics (scientific taste) seed the database: Compute can check forty thousand genes, yet it has no scientific taste to judge which biological paradox or clinical bottleneck matters. When collaborators choose the topics they want to watch, those questions establish the founding topic graphs of our living Evidence Capsule database on Nexitia.
- Original ideas guide causal triangulation: Collaborators contribute the mechanistic hypotheses and paradoxes to explore across orthogonal dry-lab datasets [7], directing the engine toward questions where resolving a contradiction changes the next study or grant application in the grant-write-master skill.
- Review comments upgrade the verification nodes: When a collaborator audits a manuscript or an Evidence Capsule synthesis on the Nexitia canvas and spots a subtle biological confounder, an assay artefact or an interpretive gap, that critique does more than fix one paper. A researcher can encode the check into a custom node (which holds the researcher's own code for any situation) or refine an AI node (which comes with a customised prompt, skill and context loading, and a gated hook for quality control) in the scientific-writing skill or the academic-peer-review skill, and name and describe that node so their own agent can reuse it.
Shared through Nexitia Core (listed today as AROS Core) and the shared-brain federation, such a check carries one scientist's careful reading forward, as a rule that another laboratory can inspect and reuse.
References
- Simpson SG (2009). Subsystems of Second Order Arithmetic, 2nd ed. Perspectives in Logic, Cambridge University Press. doi:10.1017/CBO9780511581007. Defines second-order arithmetic , quantifier-alternation complexity and the five principal subsystems (, , , and ).
- Nuijten MB, Hartgerink CHJ, van Assen MALM, Epskamp S, Wicherts JM (2016). The prevalence of statistical reporting errors in psychology (1985–2013). Behavior Research Methods 48(4): 1205–1226. doi:10.3758/s13428-015-0664-2. Ran the automated
statcheckR package on 30,717 HTML articles from eight psychology journals, extracting 258,105 APA-formatted null-hypothesis tests across 16,695 articles: 49.6% of those 16,695 articles (8,273) contained at least one inconsistent -value and 12.9% (2,150) contained at least one gross inconsistency that crossed the significance threshold. - David S (2026). Erroneous reports of protein expression in cell lines with genetic deletions and mutations. Zenodo (21 October 2026). doi:10.5281/zenodo.23498482. Checked 20 commonly used cell lines with confirmed homozygous deletions or biallelic inactivating mutations across 10 proteins against Cellosaurus, the Human Protein Atlas, Cell Model Passports and GEO sequencing, finding 230 erroneous Western blot reports across 221 published papers in a one-person search.
- Hsiao T-K, Schneider J (2021). Continued use of retracted papers: temporal trends in citations and (lack of) awareness of retractions shown in citation contexts in biomedicine. Quantitative Science Studies 2(4): 1144–1169. doi:10.1162/qss_a_00155. Analysed 7,813 retracted PubMed papers, 169,434 iCite citations and 48,134 in-text citation contexts from the PubMed Central Open Access Subset: retraction did not alter how papers were cited, and among 13,252 post-retraction citation contexts only 5.4% (722) acknowledged the retraction.
- Lowe D (2026). The case of the should-be-missing proteins. Science, In the Pipeline, 21 October 2026. https://www.science.org/content/blog-post/case-should-be-missing-proteins. Commentary on David's preprint noting that 230 erroneous Western blot examples were found in 221 papers by one person and warning that AI systems built to synthesise the published literature will ingest these errors.
- Ioannidis JPA (2016). The mass production of redundant, misleading, and conflicted systematic reviews and meta-analyses. The Milbank Quarterly 94(3): 485–514. doi:10.1111/1468-0009.12210. Surveys PubMed indexing and empirical evaluations from 1986 to 2015: between 1991 and 2014 annual publications rose 2,728% for systematic reviews and 2,635% for meta-analyses versus 153% for all PubMed items, including 185 meta-analyses of antidepressants for depression in 2007–2014 and widespread candidate-gene meta-analyses pooling fragmented, biased primary studies.
- Lawlor DA, Tilling K, Davey Smith G (2016). Triangulation in aetiological epidemiology. International Journal of Epidemiology 45(6): 1866–1886. doi:10.1093/ije/dyw314. Defines causal triangulation as comparing results across at least two approaches with unrelated key sources of bias (and ideally opposite bias directions) while accounting for exposure duration and timing, illustrated across systolic blood pressure and coronary heart disease (4.6-year RCTs, 13.2-year cohorts and 54.9-year genetic instrumental-variable studies), maternal fasting glucose and birthweight, and breastfeeding and later body mass index (where negative-control outcomes exposed residual confounding).
- Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, Alonso-Coello P, Schünemann HJ; GRADE Working Group (2008). GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ 336(7650): 924–926. doi:10.1136/bmj.39489.470347.AD. Outlines the GRADE system for stratifying evidence into four quality levels (high, moderate, low and very low) starting from study design and modifying for study limitations, inconsistency, indirectness, imprecision, reporting bias, effect magnitude and dose–response gradients.
- Shojania KG, Sampson M, Ansari MT, Ji J, Doucette S, Moher D (2007). How quickly do systematic reviews go out of date? A survival analysis. Annals of Internal Medicine 147(4): 224–233. doi:10.7326/0003-4819-147-4-200708210-00179. Survival analysis of 100 quantitative systematic reviews of randomized or quasi-randomized trials (median 13 studies and 2,663 participants): 57% developed a quantitative or qualitative updating signal, with a median survival of 5.5 years, while 23% went out of date within 2 years, 15% within 1 year and 7% by the date of publication.
- Kataoka Y, Banno M, Tsujimoto Y, Ariie T, Taito S, Suzuki T, Oide S, Furukawa TA (2022). Retracted randomized controlled trials were cited and not corrected in systematic reviews and clinical practice guidelines. Journal of Clinical Epidemiology 150: 90–97. doi:10.1016/j.jclinepi.2022.06.015. Meta-epidemiological study of 587 systematic reviews (525) and clinical practice guidelines (62) citing retracted RCTs: 0 of 127 syntheses that cited an already-retracted RCT without caution corrected themselves after publication, and among 239 syntheses published before the RCT's retraction only 5% of systematic reviews (9 of 196) and 5% of guidelines (2 of 43) corrected or retracted their results.
- Fanelli D, Wong J, Moher D (2022). What difference might retractions make? An estimate of the potential epistemic cost of retractions on meta-analyses. Accountability in Research 29(7): 442–459. doi:10.1080/08989621.2021.1947810. Examined 229 meta-analyses published in 2013–2016 that cited a retracted study: 68% of the retractions occurred at least a year before the citing synthesis appeared, only 13% of those avoidable citations (21 of 156) excluded the study as retracted; in the 50 meta-analyses that had pooled the retracted study, removing it changed the summary estimate little on average, and more when the retraction concerned data or results.
- Bakker C, Boughton S, Faggion CM, Fanelli D, Kaiser K, Schneider J (2024). Reducing the residue of retractions in evidence synthesis: ways to minimise inappropriate citation and use of retracted data. BMJ Evidence-Based Medicine 29(2): 121–126. doi:10.1136/bmjebm-2022-111921. Reviews why retracted studies persist in evidence syntheses (inconsistent database tagging, a 33-month average lag to retraction, and missing propagation into review workflows) and documents cases where excluding flawed or fabricated data changed conclusions, including a COVID-19 ivermectin mortality finding and over half of the analyses across 22 apixaban meta-analyses.
- Elliott JH, Turner T, Clavisi O, Thomas J, Higgins JPT, Mavergames C, Gruen RL (2014). Living systematic reviews: an emerging opportunity to narrow the evidence-practice gap. PLoS Medicine 11(2): e1001603. doi:10.1371/journal.pmed.1001603. Proposes living systematic reviews—continually updated online syntheses driven by continuous search, semi-automated screening, structured linked-data repositories and, as options under discussion, sequential or Bayesian methods to handle false positives across repeated updates.
- Yamada Y, Lange RT, Lu C, Hu S, Lu C, Foerster J, Clune J, Ha D (2025). The AI Scientist-v2: workshop-level automated scientific discovery via agentic tree search. arXiv:2504.08066. Submitted three autonomous manuscripts to blind peer review at an ICLR 2025 workshop: one paper reporting negative results on compositional regularisation scored 6.33 (top 45%, above the acceptance threshold), while the authors' own audit of its code and text found a 57% train–test overlap, an inaccurate Figure 3 caption and prose contradicting Figure 5.
- Si C, Yang D, Hashimoto T (2024). Can LLMs generate novel research ideas? A large-scale human study with 100+ NLP researchers. arXiv:2409.04109; ICLR 2025. Blinded study with 49 expert idea writers and 79 expert reviewers (298 reviews): every tested LLM evaluator had lower ranking agreement with the experts (43.3% to 53.3%) than the experts had with one another (56.1%).
- Wong CH, Siah KW, Lo AW (2019). Estimation of clinical trial success rates and related parameters. Biostatistics 20(2): 273–286. doi:10.1093/biostatistics/kxx069. Path-by-path analysis of 406,038 clinical trial records covering 21,143 compounds from 2000 to 2015: across 41,040 industry development paths (15,102 drugs) entering Phase 1 across all indications, an estimated 13.8% eventually reached regulatory approval.
- Tao T (2026). Mathematics in the age of AI. Proceedings of the International Congress of Mathematicians 2026 (submitted); arXiv:2608.16753. Deconstructs mathematical problem solving into five stages—proof generation, verification, exposition, community digestion/acceptance and canonicalization into definitive theory (Goal 6.5, Figure 4)—and analyses how the transition from proof scarcity to AI-driven proof abundance creates "proof indigestion" when generation outruns human verification, exposition and peer review.
- Tao T (2026). Terence Tao on AI in mathematics (and beyond): a living, curated summary of my current thinking on AI, with a companion interview (updated 2 November 2026). https://teorth.github.io/tao-web/ai-views.html. Synthesises Tao's lectures and commentary on formal verification in Lean, "Math 2.0", his proposed programme for an era of proof abundance after the "Math 1.0" of proof scarcity, OpenAI's October 2026 bulk release of 722 solved problems across more than 350 findings, and his proposals to assign priority and scientific credit to researchers who steward, verify and explain results to publication standard. The page states that it was compiled and drafted by an AI assistant (Claude) from Tao's public writing, talks and interviews and then reviewed and corrected by him.
- Greenberg SA (2009). How citation distortions create unfounded authority: analysis of a citation network. BMJ 339: b2680. doi:10.1136/bmj.b2680. Maps 242 papers and 675 citations around one claim about β-amyloid in inclusion body myositis: 94% of the 214 citations to the ten primary-data papers went to the four that supported the claim, and the six that weakened or refuted it received 6%; the study classifies the citations and does not test whether the claim is true.
- Errington TM, Denis A, Perfito N, Iorns E, Nosek BA (2021). Challenges for assessing replicability in preclinical cancer biology. eLife 10: e67995. doi:10.7554/eLife.67995. Of 193 experiments from 53 high-impact cancer biology papers, none could be designed from the paper and its supplementary materials without clarifications; 50 experiments from 23 papers were completed.
Our records:
- Biomedical Literature Evidence 1.0.0-public.1 validation record: 19 unittest methods passed; separates studies from overlapping cohort articles; 3 of 3 RIS and CSL-JSON records retained as unverified; synthetic completed-insufficient ledger check (
completed: true, support_ready: false). - Mendelian randomization 1.0.0-public.2 reproduction report: Walker et al. 2019 v2, case 2 (157 of 163 archived rows retained, 19 of 19 headline cells matched, prose-versus-Table-4 discrepancy recorded).
- Meta-analysis 1.0.0 performance report: 28 open-access Cochrane reviews, 1,060 of 1,061 pooled analyses and 10,797 of 11,351 checkable quantitative items reproduced (95%).
- Scientific Figure Master 1.1.2 skill page: 61 of 61 scored caption fields correct across 20 real captions; no significance star drawn when a recomputed contradicts the caption.
- Grant Writing Master 7.0.0-public.1 validation record and fact-preservation record: null pilot results () and non-causal wording preserved across revisions and languages.
- Scientific Writing 1.1.0 release notes: complete manuscript revision comparison with separate editorial decision record and numeric source-table verification.
- Academic Peer Review 1.0.2 skill page, benchmark and PeerJ 20995 blind review: calibrated against 1,035 comments from 60 open reviews; caught 6 of 49 row-level reviewer points and demanded 5 of 9 v1-to-v2 revisions on a blind submission.
- CohortDAG 0.2.0 release notes, Tracyte and AXIO: verified workflow nodes and visual canvas inspection.
- Nexitia Core (listed on the products page as AROS Core), shared-brain statistics, the AROS white paper, privacy policy, MCP server and curated exchange: federated procedural memory, progressive context disclosure and workflow exchange; topic-scoped living Evidence Capsule database (no public record yet).