Definitions
The vocabulary used in every record and statistic. Metric definitions version 1.1.
Findings
Each investigated claim has exactly one finding. A paper with several claims can have several different findings; there is no verdict on a paper as a whole.
- No defect found
- A deep audit within a stated scope found no defect. This is not a proof that the claim is correct.
- False as stated
- The claim, read exactly as printed with its stated assumptions, is false. A counterexample is given.
- Proof incomplete
- The proof does not establish the claim as written. The claim itself is not shown to be false, and a repair may exist.
- Missing assumption
- The claim holds only under an assumption that the paper does not state.
- Construction fails
- A mechanism, algorithm or other construction does not do what the paper says it does.
- Unresolved
- The question could not be settled, usually because the relevant version of the source could not be obtained. Not counted as a defect.
A short qualifier sometimes follows the label:
- repairable gap: a short argument or condition closes the gap in the cases checked.
- unsupported citation: the proof relies on a cited result that does not cover the case.
- previously reported: the error was already corrected or reported in print before the audit.
- source not available: the version of the source needed to settle the question could not be obtained.
Records also say which result a defect affects and what the finding does not establish.
Statuses of a candidate defect
- Candidate
- Any possible defect reported by any check. Most candidates come from AI-assisted discovery runs and are never published by name.
- Open
- Not yet settled. A candidate supported only by AI rechecks stays open until a human has checked it.
- Confirmed
- At least two confirming checks of different kinds, at least one by a human, and no refuting check left unaddressed. Only confirmed defects are published as defects.
- Unresolved
- Checks were completed but could not settle the question. Never counted as a defect or as rejected. An unresolved question appears by name only inside a record that is released for other, settled claims in the same paper; a paper is never named because of an unresolved question alone.
- Rejected
- A check showed the suspected defect does not exist. Counted, never named.
- Withdrawn
- A released record we no longer stand behind. The page stays online with the reason.
Kinds of check
Verification distinguishes a separate AI recheck from four human-support categories. A private human confirmation records only that the confirmation standard was met; it does not disclose the source or correspondence.
- Separate AI recheck
- An AI model in a new session. Never counted as a human check.
- Reproduced by the researcher
- Re-derived by hand or with separately written code.
- External human verification
- Someone outside the programme, in their own name.
- Written author agreement
- The authors agreed in writing.
- Qualifying human confirmation (private)
- A human confirmation is recorded in the private research file without disclosing its source or correspondence.
Rerunning the original evidence package shows the package works; it is reported separately and is not counted as a check.
Source check
The decisive elements are the statement, every definition and assumption the finding relies on, the decisive step, and any supplement the claim depends on.
- Full: every decisive element checked on rendered pages of the published version
- Partial: some decisive element checked only in extracted text, or not checked
- None: checked against a preprint or working paper only
Corrections
A correction is described as crediting this audit only if its own text says so. A correction published before the audit marks the finding as previously reported; it is not an outcome of this audit.
Metrics
| ID | Label | Unit | Counting rule | Updated |
|---|---|---|---|---|
| M01 | Papers considered | paper | Papers in the programme with any screening, including metadata-only screening. | monthly |
| M02 | Papers investigated at theorem level | paper | Papers with at least one claim checked against a source text or deeply audited. | monthly |
| M03 | Claims investigated | claim | Claims checked against a source text or deeply audited. | monthly |
| M04 | Claims deeply audited | claim | Claims that received a deep audit. | monthly |
| M05 | Candidate defects raised | claim | Claims for which any check ever reported a possible defect. Never decreases. | monthly |
| M06 | Status of candidate defects | claim | M05 split into open, confirmed, unresolved and rejected. | monthly |
| M07 | Deep audits that found no defect | claim | Deeply audited claims with no candidate defect ever raised, by source check. | monthly |
| M08 | Confirmed: rechecked by a separate AI session | claim | Confirmed claims with a confirming AI recheck in a separate session. | monthly |
| M09 | Confirmed: reproduced by the researcher | claim | Re-derived by hand or with separately written code. Package reruns do not count. | monthly |
| M10 | Confirmed: verified by an external human | claim | Verified by someone outside the programme. | monthly |
| M11 | Confirmed: source check | claim | Confirmed claims by full, partial or no check against the published version. | monthly |
| M12 | Papers whose authors were contacted | paper | Contact is optional and is not a publication requirement. | quarterly |
| M13 | Authors' position on confirmed findings | claim | Confirmed claims in papers whose authors were contacted, by the authors' latest written position. | quarterly |
| M14 | Corrections published after author contact | claim | Released records only; credited to this audit only if the correction says so. | monthly |
| M15 | Records released | record | Records with status published. | monthly |
| M16 | Our corrections and withdrawals | record version | Released versions that correct our error or withdraw a record. | monthly |
| M17 | Calibration checks | claim | Claims whose error was already corrected in print before the audit, used to test the method. | monthly |
A ratio is shown only when its numerator is a subset of its denominator, as counts ("6 of 11"). No ratio uses papers or claims examined as its denominator. Papers are chosen deliberately, not sampled at random. These counts describe this programme's work and cannot be read as error rates in published research.