AI for Research Efficiency, In Practice: Let adversarial reviewers check your paper
Narration: an AI-generated voice (ElevenLabs).

[0:00] A slip of ten
In twenty twenty-four, a paper on black plastic kitchen utensils multiplied seven thousand by sixty, and printed forty-two thousand.
The answer is four hundred and twenty thousand. Later, a model asked to carefully check the math in this paper found the slip.
We planted thirteen errors in a practice paper. Reviewer agents found all thirteen, in every one of six runs, in two tools.
Here is how, and what that does not prove.
AI for Research Efficiency, In Practice. Let adversarial reviewers check your paper.
In this film, four reviewer agents with one job each read your manuscript before you submit it, and hand you a ranked list of problems to judge.
They change nothing, and you will see what they found, what they got wrong, and what this test cannot show.

[1:00] 1. Ask for the process
Every film on this shelf starts the same way: ask for the process first.
What is the best way to check my manuscript before I submit it? Do not check it yet.
In plan mode, the agent interviews you first, one question at a time.
It asked fifteen, and challenged one of our claims at once: without lowering yield says more than a test that found no difference.

[1:26] 2. What already exists
Three helpers go looking.
The first finds the free tools. stat check recomputes p-values from test statistics; on what it finds, its accuracy is ninety-six point two to ninety-nine point nine percent.
But in psychology journals it finds only about sixty percent of tests, and usually none in tables.
It needs R, which our server lacks, so it was not run here; the skill does its arithmetic in Python.
Grim asks whether a mean is even possible for the number of people who answered.
Crossref flags retracted papers, free.
The second reads the research on models as reviewers. In an early study, asking for the errors worked better than asking for a review.
The third works out how to keep the manuscript unchanged, and warns that only a setting can promise that.

[2:16] 3. The options, attacked
Then a fourth helper attacks the options.
Its sharpest point: the evidence does not show that a team of reviewers beats one, so test the design on known errors and on sound papers.
Our own reading agrees. In a 2025 benchmark of ninety-one real errors in published papers, no model found more than twenty-one point one percent.
By its strict count, at most six point one percent of their flags were right, and run again, a model rarely found the same errors.
Models also invent references, and they reach print: in early twenty twenty-six, one paper in two hundred and seventy-seven in a large biomedical archive cited at least one.
And a real reference may not say what it is cited for. So every number is checked in code, and every reference is looked up and read.

[3:09] 4. Four reviewers, one job each
You choose four reviewers, each in its own context, with a single job.
A claim checker tests every number against the tables and the data.
A statistics skeptic recomputes every test, in code.
A citation checker looks each reference up, reads it, and calls it verified, a problem, or could not verify.
And Reviewer Two, the only reviewer allowed opinions, goes after overclaims and asks what is new.
Each writes only its own list of flags, with the evidence for each. A last step merges and ranks them.
The agent drafts it as a skill, in the open Agent Skills format.
We had written our own version before this session; that is the one that ran, and the page lists every difference.

[3:57] 5. Lock the copy, and test the lock
Before a run, the manuscript is copied into a review folder, its checksums are recorded, and the copy is made read-only.
Then we asked each tool to fix one word in that copy.
Claude Code's permission rule refused. Codex unlocked the file, changed it, and locked it again. Only the checksums showed it.
A permission profile stopped it, until Codex's automatic reviewer let it step outside the sandbox. When nothing could approve that step, the copy held.
Both guards also block making the copy, so you make it first.

[4:33] 6. Run it in two tools
Then the same skill ran in Claude Code and in Codex, three times each, on a fresh copy every time. Antigravity was not part of this test.
This Claude Code run checks the copy against its checksums, starts four reviewers at once, and returns a ranked list in about eight minutes.
In Codex, the skill first asked whether the survey data could be read, and a run took about fifteen minutes.
In every run, the checksums show the manuscript unchanged.

[5:07] 7. Found and missed
Now the answer key, which the agents never saw.
All six runs found all thirteen. Two of the Codex runs raised the overclaim only as Reviewer Two's opinion, which we count.
Of eighty-nine flags in all, seventy-nine pointed at planted errors, eight were real problems we had not planted, and two were false alarms, both suggestions.
We also planted three decoys that look wrong and are right. None drew a flag.
So runs of one tool agreed on all thirteen; beyond them, what else they raised differed.
But this paper was built for this skill: one error of each kind its checks look for. On real errors, recall the benchmark: no model above twenty-one point one percent.

[5:55] Another field: a real paper
Now a real paper: a 2013 psychology paper in Plos One, openly licensed, whose studies include moral-judgment scenarios.
Its authors later corrected five errors. In our own check every p-value matches its statistic, so our p-value script passes it.
With the web switched off, so the corrections stay out of reach, both tools found all five.
Each came from the printed numbers: a test its means cannot give, degrees of freedom its sample cannot have, a mean no whole-dollar total makes.
A model may have read those corrections in training, so this test is weaker than the practice paper.
And one flag was simply wrong: a reviewer's own script treated a one-to-a-hundred rating as if it ran to five.

[6:45] The line
Use this on your own manuscript, with your co-authors' agreement, and disclose it as your journal asks.
Never on a manuscript or proposal sent to you for review, unless that journal or funder explicitly allows the tool: Elsevier, Wiley, Taylor and Francis, Science, Plos and the N I H say no.
And expect false alarms: a few wrong flags are a fair price for one real error.

[7:14] Try it
Pause here and try it. The skill and the practice manuscript are on the companion page, with the answer key as a separate file.
Run the skill, read its ranked list, then open the key and count what it found and what it missed.

[7:33] Review and next
Ask for the process. Give each reviewer a single job. Lock the copy, and test the lock. And judge every flag yourself.
Next on the shelf: check your proposal against the rules.