Agentic AI in Science: Why Verification Changes the Result

Automated search can produce discoveries when candidates meet explicit checks
Scientific validity extends beyond benchmark scores and manuscript acceptance
External verification must connect generated claims to evidence and context

Agentic AI's scientific value depends partly on what happens after it proposes an answer. A system connected to an effective evaluator can reject faulty candidates and refine promising ones. A system producing fluent prose without checking its sources can circulate convincing mistakes. AlphaEvolve illustrates the first arrangement: it discovered a procedure for multiplying two four-by-four complex-valued matrices using 48 scalar multiplications, improving on the previously known method in that setting. Fabricated-reference audits illustrate the second problem, although they concern different systems and cannot be treated as evidence that the same agent produced both outcomes. The useful comparison is between verification arrangements. Science Forum should ask which claims can be checked automatically, which need contextual judgment and how those stages can be connected within a credible research process.

Search Works Better When Success Can Be Checked

AlphaEvolve combines model-generated proposals with an evolutionary search process and task-specific evaluation. The important feature is the loop: candidate code is tested, results are compared and successful variants inform further search. For a mathematical procedure, correctness and the claimed operation count can be examined explicitly. This does not make every implementation or application equally simple to assess, but it provides a strong route from suggestion to verified improvement. The model's fluency is not the evidence that the procedure works. The evaluator and inspectable result supply that evidence. This architecture helps explain why progress in a bounded search problem cannot automatically be generalised to open-ended scientific investigation.

Other agentic systems use simulations, test suites or multiple roles to improve candidate solutions. The research discussed here describes large gains on selected scientific-machine-learning benchmarks and an AI-generated manuscript receiving a favourable workshop assessment. These outcomes require different interpretations. An improvement against a particular baseline may be useful without establishing superiority across a field. A workshop score is a human evaluation under specified conditions, not an automatic proof of scientific validity. Multiple agents also do not guarantee independent scrutiny when they share similar errors. Research should identify the comparison, the evaluation process and the result's scope. The architecture creates opportunities for verification, but the scientific strength of the output depends on whether those opportunities are used effectively.

The growth of AI-assisted writing in the literature is a separate development. Vocabulary analyses examined here estimate extensive assistance in biomedical full texts, especially in interpretive sections. Such evidence concerns how prose may have been produced, rather than whether the underlying work was automated or the conclusion validated. It should not be grouped with algorithmic search as if both outcomes passed the same mechanical check. A useful distinction is between generating a candidate, evaluating it against a defined objective and establishing its scientific meaning. Some tasks permit rapid automated assessment; others require a combination of evidence, domain knowledge and independent criticism. The research process should make those boundaries visible.

Where Scientific Judgment Enters

Choosing a statistical method or making a causal claim requires attention to the question, data and assumptions. A procedure can run successfully and still answer the wrong question. A benchmark can reward a narrow target that omits important consequences. These difficulties affect human research as well as AI-assisted work, but automation can produce many plausible analyses before someone examines their relevance. The appropriate response is to specify what must be checked: data provenance, measurement, assumptions, robustness and alternative explanations. Current performance on one discovery benchmark cannot establish a permanent boundary on what models will be able to do. It can reveal where the tested system needs stronger support and where a generated result remains an unverified proposal.

Novelty and significance pose related problems. A system may propose an unfamiliar combination without knowing that specialists have already explored it, or identify a technically new result of little importance. Searchable literature helps, but the research record is incomplete and the meaning of a contribution depends on its field. AI can support comparison and generate questions for review; expert assessment may still be needed to interpret what follows. Novelty, correctness and significance should be evaluated separately. An original idea can be wrong, a correct result can be incremental and an important question can lack an immediate solution. Treating these distinctions explicitly is more useful than assigning a single score to an apparently scientific manuscript.

Citation Integrity as a Verification Problem

References provide a concrete example of a checkable claim that may remain unchecked in a writing workflow. Audits of published biomedical citations, controlled tests of generated references and investigations of conference papers examine different populations. They should not be combined into one prevalence estimate. The rates also require explicit denominators: a share of papers containing at least one fabricated reference is different from a share of all references that are fabricated. The consistent concern is that plausible bibliographic strings can reach a manuscript without a real supporting source. That is a verification failure whose consequences extend beyond formatting. Readers may follow a nonexistent trail or assume that a disputed claim has evidence it does not possess. Citation checks should therefore be integrated into research preparation and review, while maintaining a clear account of what each check establishes.

Citation verification has several stages. A reference must resolve to a real work; its author, title and publication details must identify that work correctly; and its contents must support the claim made. Automated database checks can assist with the first two, while the third often requires reading and interpretation. A real identifier attached to a misleading claim is still an unreliable citation. Retrieval-based systems can reduce some errors without eliminating them. The lesson is therefore architectural rather than fatalistic: generated references should pass through checks appropriate to what they assert. A model's internal confidence is not a substitute for a retrievable source, just as a candidate algorithm's confident description is not a substitute for a correctness test.

Designing Assistance Around the Claim

Tasks should be matched to evaluation methods rather than assigned blanket trust. Algorithm search may use formal identities or strong tests. Simulation-based optimisation needs scrutiny of the simulation's assumptions and relation to the target system. Code generation can use tests, but passing a limited suite does not establish correctness for every input. Literature synthesis requires source verification and fair representation. These are different forms of evidence, with different residual uncertainties. Researchers should describe which checks were performed and what remains outside them. The resulting division of work can make AI useful without pretending that success in a bounded task establishes general scientific competence.

A well-designed research workflow can give agents room to explore while retaining control over consequential claims. It may require source-backed references before a draft is accepted, reproducible analysis before a result is reported and domain review before a causal interpretation is endorsed. Failures should be recorded so that the evaluation itself can improve. The cost of checks also belongs in the assessment of productivity: rapid generation followed by extensive correction may save less time than the initial demonstration suggests. The aim is to make the complete route from proposal to defensible finding more effective. AlphaEvolve offers an instructive example of integrating evaluation into search; other scientific tasks need verification suited to their own uncertainties.

Agentic AI can contribute to discovery when its proposals are connected to evidence that survives scrutiny. Some domains permit fast, explicit checks; others involve incomplete observations and judgments that remain contestable. Scientific institutions should investigate that variation rather than assume either universal reliability or universal incapacity. The important unit of evaluation is the complete system of generation, checking and interpretation. For Gordon Institute's Science Forum, the frontier is therefore more than the number of ideas an agent produces. It is the growing range of claims for which humans and machines can establish a credible chain of justification, while acknowledging the limits of what has actually been demonstrated.

This Science Forum article is adapted from material published within Gordon Institute’s network. It presents editorial analysis and does not constitute professional advice or an official institutional policy.

References

Ansari, N. (2026) 'Compound deception in elite peer review: a failure mode taxonomy of 100 fabricated citations at NeurIPS 2025', arXiv preprint.
Bao, H., Sun, M. and Teplitskiy, M. (2025) 'Where there's a will there's a way: ChatGPT is used more for science in countries where it is prohibited', Quantitative Science Studies.
Gibney, E. (2025) 'DeepMind AI algorithm beats 56-year-old record for matrix multiplication speed', Nature. London: Springer Nature.
Holzwarth, L., González-Márquez, R. and Kobak, D. (2026) ‘Most biomedical publications show signs of LLM-assisted writing’, arXiv:2608.10715.
Jiang, X. and Karniadakis, G.E. (2025) AgenticSciML: Multi-Agent Systems for Physics-Informed and Operator Learning. Providence: Brown University.
Linardon, J., Jarman, H.K., McClure, Z., Anderson, C., Liu, C. and Messer, M. (2025) 'Influence of topic familiarity and prompt specificity on citation fabrication in mental health research using large language models: experimental study', JMIR Mental Health. Geelong: Deakin University.
Topaz, M., Roguin, N., Gupta, P., Zhang, Z. and Peltonen, L-M. (2026) 'Fabricated citations: an audit across 2.5 million biomedical papers', The Lancet, 407.
Yamada, Y., Lange, R.T. and Lu, C. (2025) The AI Scientist-v2: Workshop-Level Automated Scientific Discovery. Tokyo: Sakana AI.