In Practice: Check your proposal against the rules
Before you submit, have an AI agent check your own proposal against the solicitation and the funder's current guide: it builds a table of rules from the documents, quotes each one word for word, measures your PDFs (pages, type, margins, web addresses) and compares the budget with its justification. It never edits the proposal or writes its content. One real run in Codex on a fictional NSF CAREER proposal made for the film, scored against eight problems we planted: what NSF's system would also stop, and what only a person catches today.
Downloads
AI for Research Efficiency, In Practice, with Douglas Hutchings. Narration: an AI-generated voice (ElevenLabs).
Before you submit, have an agent check your own proposal against the funding opportunity and the funder's current guide. It builds a table of rules from the documents, proves that each rule's quotation is word for word, measures your PDFs, compares the budget with its justification, and labels each finding: the system would also stop this, or only a person catches this today. It never edits your files and never writes a word of your proposal. The film shows one real run in Codex, replayed, on a practice proposal made for the film. Any of the three command-line agents can run the same skill.
1. What you need
A command-line agent: Codex, Antigravity or Claude Code (episodes 2.1 to 2.3), on a monthly plan or an API key.
Python 3.10 or later with pdfplumber, in a virtual environment in your working folder:
Windows (PowerShell) macOS (Terminal) py -3 -m venv .venvpython3 -m venv .venv.\.venv\Scripts\python.exe -m pip install "pdfplumber>=0.11,<0.12".venv/bin/python -m pip install 'pdfplumber>=0.11,<0.12'Your proposal's files as you will upload them, the funding opportunity, and the funder's guide, saved that day in a
rulesfolder. The agent checks online that they are current.A tool your campus approves for unpublished ideas (episode 9).
2. The skill: check-my-proposal
Download the folder: check-my-proposal.zip. It is the skill the film's run used: the agent wrote it in plan mode (session A), the person read it beside a checklist of their own and asked for eight changes, and the agent rewrote it. Put one copy in .agents\skills\check-my-proposal\ (Codex and Antigravity) and another in .claude\skills\check-my-proposal\ (Claude Code), in the folder you will work in; on Windows, a copy, not a link. (On the Mac, the same folders with forward slashes.)
check-my-proposal/ SKILL.md the process: the lines it keeps,| Tool | How to run it | The careful mode first |
|---|---|---|
| Codex | $check-my-proposal, or ask for a proposal check | /plan (or Shift+Tab), read-only; then "Ask for approval" |
| Antigravity | /check-my-proposal, or ask for a proposal check | its default mode (not tested in this film) |
| Claude Code | /check-my-proposal, or ask for a proposal check | plan mode (Shift+Tab), or manual approvals |
Let it write only in a check folder. Tested in this film in Codex 0.164.0 only.
3. The two scripts by hand
| Windows | macOS |
|---|---|
.\.venv\Scripts\python.exe .agents\skills\check-my-proposal\scripts\quote_check.py --rules check\rules.json --out check\quote-proof.json | .venv/bin/python .agents/skills/check-my-proposal/scripts/quote_check.py --rules check/rules.json --out check/quote-proof.json |
.\.venv\Scripts\python.exe .agents\skills\check-my-proposal\scripts\pdf_evidence.py --rules check\rules.json --out check\pdf-results.json | .venv/bin/python .agents/skills/check-my-proposal/scripts/pdf_evidence.py --rules check/rules.json --out check/pdf-results.json |
A quotation the first script can't find stops every finding that rests on it: a made-up rule is worse than a missed one. The second prints PASS or FAIL with the rule's own words for each measurable rule (pages, fonts and sizes, lines per inch, margins, web addresses, headings on their own line, budget arithmetic), and MANUAL for what a person must read.
4. The practice proposal
The practice folder, proposal-practice.zip: an NSF CAREER proposal made for this film, from a fictional PI at a fictional university, written with an AI model's help and never submitted; every page says so. Eleven PDFs (the cover sheet and budget as entered in Research.gov, the Project Summary, the Project Description, References Cited, the budget justification, facilities, the mentoring plan, the Departmental Letter and two letters of collaboration), a README and a MANIFEST with checksums. Licence: CC BY 4.0. It has problems in it, and things that look wrong but the rules allow. Run the skill on it before you open the answer key.
The answer key, on its own link: the eight planted problems (file and page, the rule's section and words, and whether NSF's published list says Research.gov would stop it), the eleven decoys, and two true observations found after the run (no destination for the national meeting; a letter's hook half a point into the right margin on two pages).
5. In the film's run (28 October 2026)
Four sessions in Codex CLI 0.164.0, each in its own folder outside any repository, recorded and replayed (sped up). The person's answers were written before each session; the account's plugins and connectors were off.
- Session A, the process (plan mode, read-only, approvals by the person): the request below. It asked no questions (the folder said what it was), proposed "a source-cited compliance matrix", sent three helpers in parallel (what Research.gov checks; what no system checks; how to measure the PDFs) and a reviewer ("no verified structural defect that changes the result"; two boundaries, and a false finding it turned down), then wrote the skill. The person asked for eight changes; the agent asked one question back (how to label a warning the system shows but does not stop on) and rewrote it. About 41 minutes.
- Session B, the run (a fresh session, Codex's automatic approvals): 43 minutes. It fetched NSF's documents again (unchanged), built 217 rules, proved all 217 quotations, measured the eleven files, and reported eight findings, each with its rule's words and one of the two labels.
- Session C, the second check (a fresh session): every quoted rule and the report's measurements and budget arithmetic held. About six minutes.
- Session D, another funder: the same skill against NIFA's AFRI rules: the 16-page description fits the 18 pages of a standard grant, but its 10-point type breaks the 12-point rule, it lacks the "Project Narrative" header, and a letter's hook crosses the right margin by about half a point (a strict reading). 20 minutes.
Checked against the answer key (by the studio, not the agent): found 8 of 8, none missed; the two labels agree with the key on all eight (three the system would also stop: checks 29, 35 and 36 on NSF's list of January 30, 2023; five only a person catches today); no rule made up (the studio's own script found all 277 quotations in the report in NSF's documents); none of the eleven decoys flagged.
The request, in plan mode:
What is the best way to check this proposal against everything the solicitation and the funder require?The run, in a fresh session:
Use the check-my-proposal skill on the proposal in proposal/Every prompt as typed, the person's eight changes and the full record: research/parts/run.md.
6. The rules, by version (read on 28 October 2026)
NSF plans to replace the PAPPG with a Guidance on Financial Assistance. Have your agent read the current guide and solicitation every time; never trust a rule from memory, the agent's or yours.
7. The line
- Your own proposal only. A proposal you were sent to review stays out of AI tools: NIH bars generative AI from review critiques (NOT-OD-23-149), and NSF bars reviewers from uploading proposal content to non-approved generative AI tools (December 14, 2023).
- The writing stays yours. NIH "will not consider applications that are either substantially developed by AI, or contain sections substantially developed by AI, to be original ideas of applicants" (NOT-OD-25-132). NSF encourages proposers to say in the project description whether and how they used generative AI, and counts fabrication, falsification or plagiarism through AI-based tools as research misconduct (NSF 26-200).
- Your campus's rules. Unpublished ideas may be data your campus protects (the University of Arkansas classes "Research data prior to publication" as Highly Sensitive). Use a tool your campus approves for them (episode 9).
- The grants office has the last word.
Sources
The documents above; NIH NOT-OD-25-132 (https://grants.nih.gov/grants/guide/notice-files/NOT-OD-25-132.html (opens in a new tab)); NIH NOT-OD-23-149 (https://grants.nih.gov/grants/guide/notice-files/NOT-OD-23-149.html (opens in a new tab)); NSF's notice on generative AI in merit review (https://www.nsf.gov/policies/ai/merit-review (opens in a new tab)); University of Arkansas policy 921.0 (https://policies.uark.edu/fayetteville-policies/uits/9210.php (opens in a new tab)); the Agent Skills specification (https://agentskills.io/specification (opens in a new tab)).
Corrections
None so far. If you find something wrong, email doug,@douglashutchings.com.
Transcript
Every spoken line, by chapter
Eight problems, one check
In one of the Department of Agriculture's grant programs, an application over its page limit, or outside its rules for margins and type size, is returned without review.
N S F says plainly that its submission system doesn't check every rule.
So we planted eight problems in a proposal, and asked an agent to check it. It found eight of the eight.
AI for Research Efficiency, In Practice: check your proposal against the rules.
In this film, an agent checks your own proposal against the call and the funder's current guide, before you submit.
It measures the files and quotes the rules. It never changes a file or writes a word of the proposal.
It builds on episodes four and nine, on small habits and on your data.
1. Ask for the process
The proposal is invented for this film: an N S F CAREER proposal about rice, water and greenhouse gases.
In Codex, in plan mode, the request is a question. What is the best way to check this proposal against everything the solicitation and the funder require? Do not change it.
It asks no questions. It reads the folder, opens N S F's pages online, and proposes a plan.
2. What the system stops, and what it leaves out
Then three helpers research the options in parallel.
One reads N S F's own list of what its submission system stops, like a project description over fifteen pages, or a web address inside it.
Another looks for what that list leaves out, like the letters, and whether the budget justification agrees with the budget. A third works out how to measure the type, the lines and the margins.
That second list is where the agent helps most. The first, it catches a week before the deadline.
3. The plan, attacked
A reviewer agent then attacks the plan.
The rules move. N S F's guide dates from twenty twenty-four and has been amended twice, and a replacement is planned. Its published list of automatic checks dates from January twenty twenty-three.
So the process reads the current documents every time, and names their versions.
A made-up rule is worse than a missed one, so every finding has to quote its rule, word for word.
And the agent writes none of the proposal. N I H won't count applications substantially developed by A I as the applicant's original ideas.
4. The process, written down as a skill
You choose the plan, and the agent writes it down as a skill: a file it reads, so the same steps run every time.
Then you read it beside your own checklist and ask for eight changes, and the agent rewrites the skill.
The skill builds a table of rules from the documents, each with its section and its exact words.
A script measures every P D F: pages, type sizes, lines per inch, margins and web addresses.
The agent compares the budget with its justification, and the people with their time.
Each finding gets one of two labels: the system would also stop this, or only a person catches this today.
5. The run
The run, replayed from its log: forty-three minutes of work in a copy of the folder outside any repository, sped up.
It fetches N S F's guide, its two supplements and the CAREER solicitation again, checks they match the saved copies, and writes down each version.
It builds two hundred and seventeen rules, and the quote check finds all two hundred and seventeen quotations, word for word.
The script measures all eleven files. Then the agent reads the description, the budget and the letters side by side.
Then it writes the report: each finding with its file and page, the rule's own words, and one of the two labels.
Midway through, it notes that the budget's totals add up, but the justification asks for two graduate assistants where the form budgets one, and for more travel than the form.
6. Checked against the answer key
Now the check that doesn't rest on the agent's word: the answer key, written before the run and kept out of the agent's folder.
It found eight of the eight. It missed none.
Three of the eight are on N S F's published list of checks its system makes: the sixteen pages, the heading, and the web address.
The other five aren't on that list, so you can't count on the system to catch them: ten-point type, the missing headings in Prior Support, two budget lines the justification doesn't match, and a letter that praises the proposal where the rules ask for one sentence of intent.
And its labels agree with the key on all eight.
A second agent, in a fresh session, re-checked every quoted rule and the report's measurements. Nothing failed.
And our own script found every quoted rule in N S F's documents. The report flagged none of the eleven decoys: things that look wrong, but the rules allow.
Same process, another funder
The same skill works for other funders. For the Agriculture and Food Research Initiative, the narrative can run eighteen pages, in twelve-point type.
Checked against those rules, the same project description fits the eighteen pages, but its ten-point type breaks the twelve-point rule, and it lacks the header N I F A asks for.
The line not to cross
Use it on your own proposal only. N I H bars A I from review critiques, and N S F bars reviewers from putting proposals into unapproved A I tools.
Unpublished ideas can be data your campus protects, so use a tool your campus approves. And the grants office still has the last word.
Try it
Try it on the practice proposal first. The folder and the skill are on the page, for Codex, Antigravity and Claude Code.
Review and next
Ask for the process, check every finding's quote, and run the skill on your own proposal a week before the deadline.
More films for this shelf are on the way.