Scientific publishing: no verification, no trust
How can verification become as visible as publication? The number of indexed articles grew by about 47% between 2016 and 2022 [1], peer review relies on unpaid work, and generative AI makes plausible text almost free. The answer is not more journals: it is to check the data and code behind what is published, with the same rigour Europe now demands of AI systems.
From publication to checkable evidence
- 01IdentifyDOI · authors · version
- 02ExecuteData · code · environment
- 03CompareExpected result · tolerance
- 04ChallengeAssumptions · limits · transfer
Reproducing a number does not prove that the model describes the world correctly. These levels answer different questions: documenting them separately avoids turning a badge or a repository into a blanket guarantee.
| Documented state | What is known | Open question |
|---|---|---|
| Accessible | A reader can obtain the materials. | Does the computation run? |
| Executable | The code runs in a described environment. | Does it recover the reported results? |
| Reproduced | A documented run recovers a result within a stated tolerance. | Are the assumptions and method appropriate? |
| Validated for a use | Scientific criteria and a domain of application have been examined. | What happens outside that domain? |
Connect practices without confusing their purposes
FAIR supports discovery and reuse; licences define permissions; reproducibility lets others check a computation; the AI Act regulates specified AI uses. RTVACOS connects this work to research excellence. These frameworks complement rather than replace one another.
The global picture: too many articles, too little verification
- Output that outruns researchers. Articles indexed in Scopus and Web of Science rose by about 47% from 2016 to 2022, faster than the number of scientists; part of the growth came from special issues with shortened review times [1].
- A concentrated market. Five publishers published more than half of all papers as early as 2013 [2]; the group that owns the largest reported an adjusted operating margin of 34.8% in 2025 [3].
- Invisible work. Reviewers gave more than 100 million hours in 2020 [4], unpaid and almost unrecognized.
- Eroded trust. More than 10,000 papers were retracted in 2023, over 8,000 of them in one publisher's journals, mostly in special issues exploited by paper mills [5].
- AI banned rather than framed. Rules say what not to do, for instance the ban on generative AI in US grant review [6], rarely what to show.
What already exists
| Level | What exists | What it covers |
|---|---|---|
| International | FAIR principles [8]; OSTP memo [7] | Findable, reusable data; immediate access to articles and data from US federally funded research. |
| International | JOSS [9], CODECHECK [10], ReScience C [11], SoftwareX [20] | Open software review; independent execution of an article's code; published replications; software articles and developer credit (SoftwareX). These initiatives provide different forms of review. |
| Europe | Council conclusions of May 2023 [12]; Diamond Open Access action plan [13]; Open Research Europe [14] | Immediate open access as the norm for public research; publishing at no cost to authors or readers. |
| Europe | CoARA [15] | Assessment reform: recognizing contributions other than article counts, peer review included. |
| Europe | AI Act [16] | For high-risk AI systems: data, documentation, logs, quality, post-market monitoring. |
| France | Digital Republic Act, art. 30 [17] | Right to post the accepted manuscript in open access after 6 months (12 in the humanities and social sciences). |
| France | National Plan for Open Science [18]; HAL; Software Heritage [19] | Target of 100% open-access articles by 2030; open research source code; code archiving. |
These texts open access. Almost none asks anyone to check that a published result can be reproduced from its data and code.
The AI Act as a comparison, not a regulation of papers
The AI Act requires providers of high-risk AI systems to govern their training, validation and test data (Art. 10), keep technical documentation (Art. 11), log events (Art. 12), run a quality management system (Art. 17) and monitor the system after it is placed on the market (Art. 72) [16]. These obligations overlap with some MLOps practices, without being equivalent. Scientific articles follow different editorial, scientific and sometimes regulatory rules; independent execution is not automatically guaranteed.
The regulation also excludes AI developed solely for research (Art. 2(6)). A gap may open: European teams bear the cost of compliance as soon as they deploy, teams outside the regulation do not. The AI Act remains essential all the same: it provides one framework for traceability from data to result. The right answer is not to weaken it but to bring its spirit to publishing.
France: editorial work that does not count
France has one of the most open-access-friendly legal frameworks in Europe: a mandatory right to deposit, HAL, Software Heritage, a national plan that covers source code [17] [18]. Yet editing a journal, reviewing or checking code is almost absent from career assessment, which counts articles first. The work that guarantees quality is the work that pays least; editors and reviewers become hard to find, and others end up running the reference journals.
What is missing
- Checking the data and code of every article that depends on them, as the AI Act does for AI systems: frozen versions, documented environment, independent execution, kept records.
- Declaring the tools used, AI included. Neither ban nor silence: a statement of what was used, for what, and what a human checked.
- Following the article after publication, as one monitors a deployed system: versions, errata, reports, links to corrected datasets.
- Recognizing editorial and review work in assessment, as CoARA proposes.
- Connecting the frameworks: code licences, FAIR data and the AI Act talk about the same thing, traceability, but do not cite each other.
This is not necessarily about more journals, but about changing what we ask of the existing ones. On this site, every study links to its code and data; the page AI in research shows how to declare tools, the page Data how to choose a license, the page Cite and reuse how to give credit.
No verification, no trust
An article no one can redo is a claim, not a result. The tools to check exist, AI included; what is missing is making them normal, and recognizing the people who do that work.
Questions and answers
Is scientific publishing really in trouble?
The number of indexed articles grew by about 47% between 2016 and 2022, faster than the number of researchers; unpaid peer review no longer keeps up, and more than 10,000 papers were retracted in 2023, a record.
Should AI be banned from writing and reviewing?
No: it should be declared. As with any other tool, authors state what was used and for what; what matters is that results are checked, data and code included.
Why bring the AI Act into a discussion of articles?
Because it requires of high-risk AI systems what articles lack: data governance, technical documentation, record-keeping, a quality management system and post-market monitoring.
Do we need new journals?
Not necessarily. What is missing is verification and recognition of editorial work, not more titles.
Sources
- Hanson, Gómez Barreiro, Crosetto, Brockington (2024). The strain on scientific publishing. Quantitative Science Studies 5(4). doi.org/10.1162/qss_a_00327
- Larivière, Haustein, Mongeon (2015). The oligopoly of academic publishers in the digital era. PLoS ONE 10(6). doi.org/10.1371/journal.pone.0127502
- RELX (2026). RELX 2025 results. www.relx.com/media/press-releases/year-2026/relx-2025-result
- Aczel, Szaszi, Holcombe (2021). A billion-dollar donation: estimating the cost of researchers' time spent on peer review. Research Integrity and Peer Review 6. doi.org/10.1186/s41073-021-00118-2
- Van Noorden (2023). More than 10,000 research papers were retracted in 2023, a new record. Nature 624. doi.org/10.1038/d41586-023-03974-8
- NIH (2023). NOT-OD-23-149: generative AI prohibited in NIH peer review. grants.nih.gov/grants/guide/notice-files/NOT-OD-23-149.html
- OSTP (2022). Ensuring free, immediate, and equitable access to federally funded research. bidenwhitehouse.archives.gov/wp-content/uploads/2022/08/08-2
- Wilkinson et al. (2016). The FAIR Guiding Principles for scientific data management and stewardship. Scientific Data 3. doi.org/10.1038/sdata.2016.18
- Journal of Open Source Software: open review on GitHub. joss.readthedocs.io/
- Nüst, Eglen (2021). CODECHECK: an open science initiative for the independent execution of computations. F1000Research 10. doi.org/10.12688/f1000research.51738.2
- ReScience C: reproducible replications in computational science. rescience.github.io/
- Council of the EU (2023). Conclusions on high-quality, transparent, open, trustworthy and equitable scholarly publishing (9616/23). european-research-area.ec.europa.eu/documents/council-conclu
- Science Europe, cOAlition S, OPERAS, ANR (2022). Action Plan for Diamond Open Access. doi.org/10.5281/zenodo.6282402
- European Commission. Open Research Europe. open-research-europe.ec.europa.eu/
- CoARA. Agreement on Reforming Research Assessment (2022). coara.eu/agreement/the-agreement-full-text/
- Regulation (EU) 2024/1689 (AI Act). eur-lex.europa.eu/eli/reg/2024/1689/oj
- Loi n° 2016-1321 pour une République numérique, article 30 (art. L533-4 du code de la recherche). www.legifrance.gouv.fr/jorf/id/JORFTEXT000033202746
- Deuxième Plan national pour la science ouverte 2021-2024. www.enseignementsup-recherche.gouv.fr/fr/une-politique-ambit
- Software Heritage, l'archive universelle du code source. www.softwareheritage.org/
- SoftwareX: research software publications and reuse. www.sciencedirect.com/journal/softwarex