I wrote AI slop again this morning. An agent drafted, I skimmed, and the text read smoothly enough to pass. Only on a second reading did I see what it was: tidy paragraphs that said less than they seemed to, with the same turns of phrase every few lines. It looked like slop, read like slop and smelled like slop. It belonged in the bin.

This keeps happening to researchers who use AI seriously, and this week the reason became clearer to me. A team at Meta (Jason Weston, Ilia Kulikov and Swarnadeep Saha) tested whether AI models can tell expert writing from model writing. They gave a model a published paper with one section removed, asked it to write the missing section, and then asked AI models, including the writers themselves, which version was better. The AI judges preferred the model’s section over the human original in 63.5% to 84.6% of comparisons. When the judges scored against rubrics that a model had written, they preferred the model every time. The work is a blog post and has not yet been peer-reviewed.

So the most common quality check in agentic writing, asking the AI whether the draft is good, is biased towards yes. The AI is grading its own homework, and it likes its own handwriting.

Two kinds of slop

Prose slop is the kind of writing everyone notices. In one of my own writeups, an agent’s draft used the construction “X, not Y” nine times. Add a hedge after every claim, lists of three, signposting, and sections that retell the whole paper, and a reader stops trusting the text within a page. Readers can tell it was written by AI, and they switch off. Length is the trigger. Ask a model to improve a sentence or a paragraph and it usually does it very well. Ask it to write a whole essay or paper in one go, and the result is almost always slop.

The second kind of slop is more dangerous because it looks like care. A draft article of mine said that a study’s author “cautions against reading it as proof of harm”. It sounded balanced. The paper says the opposite: its findings “provide compelling evidence for the cognitive impacts of AI tool usage”. A fabricated caveat reads as diligence, so it survives self-review by the AI agent. My own work supplies other examples. In one citation, “considers” drifted to “accounts for”, turning a design judgment into a statistical adjustment the source never described or intended. A PDF converter turned 0.00247 into 00247. When reading an image, the vision model counted 13 marks on a figure in one run and 19 in the next.

Asking the model whether its work was good would have caught none of these. Each was caught by checking against something outside the model: the source text, a second extraction engine, a pixel count and my own eyes.

A loop that learns

I use Claude Code, Codex and several open models to help with my systematic reviews, meta-analyses and papers. Since the summer I have stopped hunting for the perfect prompt and built a loop instead. I call it the agentic unslopper. It has four parts.

First, every failure becomes a lesson with a tripwire. When I catch an agent’s mistake, the lesson goes into a shared memory, a folder of short notes that every agent reads, and it must name the check that would have caught it: a search pattern, an audit script or a gate in the workflow. The invented caveat produced such a rule: any sentence saying a source “cautions”, “notes” or “warns” must be found in the source before the draft is saved. The check matters more than the note. Agents forget written advice; a check runs every time.

Second, numbers never live in the model’s memory. Every value that is extracted, derived, calculated or estimated is written to disk the moment it exists, and later steps read it from the file. This is a hard rule in the instruction file that all my agents load. Data extraction for a meta-analysis runs through two AI engines plus a visual read of the page: one engine extracts, the second verifies, and a third model re-checks the tables. Any disagreement is resolved against the original PDF. I chose this set after comparing nine extraction engines on my own papers.

Third, my text is the default. When I edit a draft and an agent reviews my edits, it may flag only what can be verified: a changed number, a certainty claim the study design cannot carry, a citation or a quotation. It may not nudge my wording back towards its own. I learnt this the hard way: I once asked an agent to check and fix the citations in a manuscript, and it also ‘fixed’ some of my prose, changing the meaning of what I wanted to convey. I made it a rule this week, after reading the Meta study. A second rule came with it: judge a draft against an expert example of the same kind, and list what is missing instead of giving a score.

Fourth, the loop feeds itself. Each workflow reads its own lessons at the start of every run, so a mistake made on Monday shapes Tuesday’s work. This is the recursive part, and the loop itself needs checking. A recent audit found 61 saved lessons about specific workflows, but only 5 in total had been wired into the workflow they concerned. The rest were orphan lessons that no workflow was reading. The agents were learning and then not using what they had learnt. A learning system decays quietly unless you audit the learning as well.

What it cost, and what it bought

The loop is slower than accepting the first draft. While preparing a perspective on AI in nutrition research for submission, three agents audited every cited claim in parallel, and each source was then checked against the original document. Of 31 cited claims, 24 were fully supported and 7 only partly. We made 13 corrections, most of them to wording that claimed a little more than the source said. None was dramatic; together they were the difference between a paper that is right and one that sounds right.

Unslopping can also overshoot. When I told an agent that a document I had written with AI’s help read like AI slop, it cut the text from 2,647 to 1,913 words and removed six of the nine “X, not Y” constructions. The result was disjointed and jarring. We restored the transitions and the useful signposts, and settled at 2,037 words. That lesson went into memory too: aim to cut 20–25%, keep the sentences that carry the argument from one point to the next, and stop there.

Where the human stays

In July I told my research group that writing is the main choke point in our research projects. AI can write methods and results in our standard laboratory style; the introduction and discussion are where we must put in our own effort. I still think so. A recent experiment by Kirgis and colleagues reaches the same place from another direction: AI agents completed all the engineering for two research projects without human help, but the projects’ original authors rejected both resulting papers.

So this is how I now write with AI. I ask the agent for an outline first. I review it, edit it and rewrite the points in my own words, then ask the agent to review it again. When typing is too slow, I switch on the microphone in Claude Code or Codex and talk through what I want to add, and the agent turns it into an outline. Only when I approve the outline do we write prose, one paragraph at a time, and I read and edit every sentence before moving on. I approve each paragraph separately in sequence. In my experience AI handles long-form writing badly: it loses the thread of the argument, and that is where slop creeps in. Reviewing and editing each line oneself is the key. The human mind drives the writing; the AI still remains just a tool that can write.

The unslopper does not take me out of the loop. My judgment is the gate: the agents stop there and ask for permission, and nothing moves forward until I decide. Because the workflows improve themselves, the output gets better with each round and my decisions get easier. The Meta team trained its model against rubrics built from expert writing. In my system the expert is me, and the rubric is every correction I have made. That is also why I cannot hand all my thinking to the agents. A judge who stops reading goes stale, and then the homework grades itself again.