Verification in Scientific Publishing: What Checkable Claims Require

Correctness, reproducibility and significance require different judgments
Mathematical proof offers lessons without making any discipline immune to error
Publishing standards should connect every important claim to an inspectable basis

Scientific publishing relies on claims that other people can examine. Mathematical proof makes this requirement especially explicit: a conclusion should follow from stated premises through defensible steps. Computational and empirical work needs its own forms of justification, including appropriate data, methods and reproducible results. Comparisons between mathematics, engineering and computing should not imply that one discipline is immune to error while others lack rigorous standards. Fabricated citations and unrepeatable results are problems across research. Science Forum's question is what makes a claim checkable and how publication processes can ensure that the necessary basis is available. AI-generated prose makes the question more urgent because the appearance of a sound argument becomes easier to produce without establishing that its evidence holds.

Proof and the Value of Explicit Justification

A proof provides a structured route for examining whether a conclusion follows. That is a valuable model of explicit justification, but mathematical practice still depends on people, conventions and sometimes difficult arguments. Errors can escape review; cited results can be misunderstood; computer-assisted proofs may introduce additional requirements for checking. Formal verification can strengthen confidence within its stated assumptions, while not settling whether a theorem is important or its assumptions relevant to an application. The lesson is therefore about making the grounds of a claim inspectable. Other disciplines can apply that principle without pretending that empirical uncertainty is identical to deductive correctness or that every useful finding should be expressed as a formal proof.

Computational results often require information about data, implementation and evaluation conditions. A benchmark score can be reproduced only if the necessary materials and procedures are available, and even successful reproduction does not establish relevance beyond the benchmark. The research discussed here describes rapid growth in submissions at AI conferences as a challenge for scrutiny. Volume can increase workload, but does not by itself establish a decline in review quality. A stronger analysis would examine the resources allocated, the checks performed and the errors discovered. Publishing standards should identify which evidence readers need to assess a reported result, including limitations that a headline performance number conceals.

The NeurIPS 2021 consistency experiment offers evidence about review decisions, rather than a direct measure of the validity of accepted science. A random sample of submissions received independent reviews, producing disagreement across committees. The reported 50.6 percent figure concerns the share accepted by one committee that would be rejected by the other, not disagreement across all submissions. Selectivity and borderline judgments affect its interpretation. These findings caution against treating acceptance as a precise ranking of quality. They do not show that correct results are selected randomly or that verification is impossible. Journals and conferences can improve clarity by separating checks on validity from assessments of novelty and significance, while acknowledging disagreement in the latter.

Citation Integrity and Empirical Evidence

Citation integrity is another form of checkability. The biomedical audit published by Topaz and colleagues reports increasing fabricated references, making verification an important part of maintaining the record. Different rates require explicit denominators: papers with a fabricated citation and fabricated references among all citations describe different quantities. Comparisons also need consistent time periods and database coverage. Vocabulary-based estimates of AI-assisted writing concern a separate phenomenon and do not establish fabrication. A reference should identify a real work, represent its details correctly and support the claim for which it is cited. Those checks matter whatever tool produced the bibliography. They can be performed at preparation and review stages, reducing reliance on a reader discovering the problem only after publication.

Neither a real experiment nor a reproducible benchmark excuses a false citation. Conversely, a faulty reference does not automatically invalidate every independent finding in the paper. Readers need to understand the role of the affected claim. A fabricated source supporting a central method may be more consequential than an incorrect bibliographic detail, but both require correction. Empirical and computational papers also differ internally in what can be salvaged when a claim fails. Broad assertions that one field retains real findings while another has nothing left are therefore unhelpful. Verification should follow the structure of the argument: which conclusions depend on which evidence, and what changes if a link does not withstand examination?

Build Verification Around the Type of Claim

A publication process can assess validity and significance separately. Reviewers may find a result sound but incremental, or interesting but insufficiently supported. Reproducibility checks can examine whether reported computations follow from the supplied materials. They should also distinguish reproducing the authors' calculation from independently replicating the finding under new conditions. Neither is feasible in exactly the same form for every field. Clear requirements can specify the level of evidence appropriate to the claim and explain what remains uncertain. The useful lesson from proof is the visibility of justification, not a demand that all disciplines adopt a single timetable or method of review. For example, an evaluator might reproduce the reported score while finding that the comparison uses an unsuitable baseline. Another might confirm the implementation while identifying leakage between training and evaluation data. These are different checks on different parts of the claim. Reporting them separately makes it easier to see what confidence is justified and what additional work is needed.

Citation checks can be built into preparation and submission workflows. Automated searches may identify unresolved references or mismatched metadata, but a missing database match is not proof that a source is fabricated. Books, older literature and non-English publications may require different methods. Human review should resolve important ambiguities and examine whether the source supports the text. The same workflow can record corrections before publication. AI assistance may help organise these checks, provided its own suggestions are verified. The objective is a reliable evidence trail that readers can follow, rather than another opaque score presented as a guarantee of integrity.

Speed and scrutiny should be evaluated together. Rapid publication can be valuable when a result addresses an urgent problem, but distributing an unsupported claim may create delays and costs for everyone who relies on it. Verification also consumes resources, and an indiscriminate demand for exhaustive review can prevent useful work from appearing. Proportionate standards should focus on consequential claims and known failure modes. Research on the review process can compare error detection, time and downstream correction under different arrangements. A transparent account of those trade-offs is stronger than either assuming that faster work is careless or that checking necessarily obstructs progress.

Independent checking should not require surrendering every legitimate interest in a dataset or implementation. Privacy, security and intellectual-property constraints can restrict public release. Institutions can sometimes provide controlled access, independent audits or sufficiently detailed methods and synthetic examples, while clearly explaining what those arrangements establish. They may not fully replace open reproducibility, and limitations should remain visible. The standard is to provide a credible basis for scrutiny suited to the claim, rather than make unrestricted disclosure the only route to publication. This allows verification to be strengthened without overlooking the ethical and practical conditions of scientific work.

Review systems also need the people and incentives to perform these checks. Requirements that appear in a policy but are not resourced may become ceremonial. Journals can specify responsibilities, use appropriate technical assistance and give reviewers access to the materials they need. Authors can prepare clear documentation and identify uncertain claims. Funders and institutions can recognise work that improves reproducibility instead of valuing only rapid output. These arrangements should be assessed through their effects: whether readers can inspect results, whether important errors are found and whether correction becomes easier. The scientific record becomes stronger through functioning practices, not through claims that a discipline has inherently superior rigor. Institutions can pilot these arrangements on representative submissions before making broad requirements. Pilots should record what the checks detect, how often they raise inconclusive concerns and how much work they add for authors and reviewers. This evidence can guide a more useful standard than a universal demand adopted without knowing whether people can carry it out.

A Standard That Preserves Both Speed and Scrutiny

Verification in publishing means connecting claims to evidence that can be examined. Proof, reproducible computation, source checking and empirical replication contribute in different ways. None eliminates every uncertainty, and none can replace judgment about relevance and significance. AI can accelerate parts of this process while also producing plausible claims that need scrutiny. For Gordon Institute's Science Forum, the appropriate response is to make justification more visible and checks more effective. A publication should explain what has been demonstrated, under which assumptions and with which limitations. That standard allows speed to support scientific progress while preserving the reader's ability to ask whether the result actually holds.

This Science Forum article is adapted from material published within Gordon Institute’s network. It presents editorial analysis and does not constitute professional advice or an official institutional policy.

References

Beygelzimer, A., Dauphin, Y., Liang, P. and Wortman Vaughan, J. (2021) 'The NeurIPS 2021 consistency experiment', Neural Information Processing Systems Blog.
Holzwarth, L., González-Márquez, R. and Kobak, D. (2026) ‘Most biomedical publications show signs of LLM-assisted writing’, arXiv:2608.10715.
Kobak, D., González-Márquez, R., Horvát, E-Á. and Lause, J. (2025) 'Delving into LLM-assisted writing in biomedical publications through excess vocabulary', Science Advances, 11(27), eadt3813.
Mascáto Fontaína, N. et al. (2025) 'Identifying common patterns in journals that retracted papers from paper mills', Research Integrity and Peer Review, 10(21).
Retraction Watch (2026) 'One in 277 PubMed-indexed papers in 2026 shows fabricated references, says analysis'. New York: The Center for Scientific Integrity.
Su, B., Zhang, J., Collina, N., Yan, Y., Li, D., Cho, K., Fan, J. and Su, W. (2026) 'Recommending best paper awards for ML/AI conferences via the isotonic mechanism', arXiv preprint arXiv:2601.15249.
Topaz, M., Roguin, N., Gupta, P., Zhang, Z. and Peltonen, L-M. (2026) 'Fabricated citations: an audit across 2.5 million biomedical papers', The Lancet, 407, pp. 1779–1781.