As Stated

Definitions

The vocabulary used in every record and statistic. Metric definitions version 1.1.

Findings

Each investigated claim has exactly one finding. A paper with several claims can have several different findings; there is no verdict on a paper as a whole.

No defect found
A deep audit within a stated scope found no defect. This is not a proof that the claim is correct.
False as stated
The claim, read exactly as printed with its stated assumptions, is false. A counterexample is given.
Proof incomplete
The proof does not establish the claim as written. The claim itself is not shown to be false, and a repair may exist.
Missing assumption
The claim holds only under an assumption that the paper does not state.
Construction fails
A mechanism, algorithm or other construction does not do what the paper says it does.
Unresolved
The question could not be settled, usually because the relevant version of the source could not be obtained. Not counted as a defect.

A short qualifier sometimes follows the label:

  • repairable gap: a short argument or condition closes the gap in the cases checked.
  • unsupported citation: the proof relies on a cited result that does not cover the case.
  • previously reported: the error was already corrected or reported in print before the audit.
  • source not available: the version of the source needed to settle the question could not be obtained.

Records also say which result a defect affects and what the finding does not establish.

Statuses of a candidate defect

Candidate
Any possible defect reported by any check. Most candidates come from AI-assisted discovery runs and are never published by name.
Open
Not yet settled. A candidate supported only by AI rechecks stays open until a human has checked it.
Confirmed
At least two confirming checks of different kinds, at least one by a human, and no refuting check left unaddressed. Only confirmed defects are published as defects.
Unresolved
Checks were completed but could not settle the question. Never counted as a defect or as rejected. An unresolved question appears by name only inside a record that is released for other, settled claims in the same paper; a paper is never named because of an unresolved question alone.
Rejected
A check showed the suspected defect does not exist. Counted, never named.
Withdrawn
A released record we no longer stand behind. The page stays online with the reason.

Kinds of check

Verification distinguishes a separate AI recheck from four human-support categories. A private human confirmation records only that the confirmation standard was met; it does not disclose the source or correspondence.

Separate AI recheck
An AI model in a new session. Never counted as a human check.
Reproduced by the researcher
Re-derived by hand or with separately written code.
External human verification
Someone outside the programme, in their own name.
Written author agreement
The authors agreed in writing.
Qualifying human confirmation (private)
A human confirmation is recorded in the private research file without disclosing its source or correspondence.

Rerunning the original evidence package shows the package works; it is reported separately and is not counted as a check.

Source check

The decisive elements are the statement, every definition and assumption the finding relies on, the decisive step, and any supplement the claim depends on.

  • Full: every decisive element checked on rendered pages of the published version
  • Partial: some decisive element checked only in extracted text, or not checked
  • None: checked against a preprint or working paper only

Author positions

Author contact is not a publication requirement. When a written author position exists, it may be recorded as agreed, partly agreed or disagreed. A record may also state that the authors were not contacted. Private correspondence is published only with separate consent to its public use.

Corrections

A correction is described as crediting this audit only if its own text says so. A correction published before the audit marks the finding as previously reported; it is not an outcome of this audit.

Metrics

IDLabelUnitCounting ruleUpdated
M01Papers consideredpaperPapers in the programme with any screening, including metadata-only screening.monthly
M02Papers investigated at theorem levelpaperPapers with at least one claim checked against a source text or deeply audited.monthly
M03Claims investigatedclaimClaims checked against a source text or deeply audited.monthly
M04Claims deeply auditedclaimClaims that received a deep audit.monthly
M05Candidate defects raisedclaimClaims for which any check ever reported a possible defect. Never decreases.monthly
M06Status of candidate defectsclaimM05 split into open, confirmed, unresolved and rejected.monthly
M07Deep audits that found no defectclaimDeeply audited claims with no candidate defect ever raised, by source check.monthly
M08Confirmed: rechecked by a separate AI sessionclaimConfirmed claims with a confirming AI recheck in a separate session.monthly
M09Confirmed: reproduced by the researcherclaimRe-derived by hand or with separately written code. Package reruns do not count.monthly
M10Confirmed: verified by an external humanclaimVerified by someone outside the programme.monthly
M11Confirmed: source checkclaimConfirmed claims by full, partial or no check against the published version.monthly
M12Papers whose authors were contactedpaperContact is optional and is not a publication requirement.quarterly
M13Authors' position on confirmed findingsclaimConfirmed claims in papers whose authors were contacted, by the authors' latest written position.quarterly
M14Corrections published after author contactclaimReleased records only; credited to this audit only if the correction says so.monthly
M15Records releasedrecordRecords with status published.monthly
M16Our corrections and withdrawalsrecord versionReleased versions that correct our error or withdraw a record.monthly
M17Calibration checksclaimClaims whose error was already corrected in print before the audit, used to test the method.monthly

A ratio is shown only when its numerator is a subset of its denominator, as counts ("6 of 11"). No ratio uses papers or claims examined as its denominator. Papers are chosen deliberately, not sampled at random. These counts describe this programme's work and cannot be read as error rates in published research.