Why AI-Writing Detectors Do Not Work — and What to Do Instead

When AI writing tools arrived, AI detectors arrived promptly behind them, promising to restore certainty: paste in the essay, receive a percentage. The promise has not survived contact with evidence. Detectors misfire in both directions — flagging honest work and missing dishonest work — and several major providers have withdrawn or heavily caveated their own tools as a result. Understanding why they fail matters, because the failure is structural, not a bug awaiting a fix.

The false positive problem

Detectors do not detect AI; they detect statistical patterns — text that is fluent, well-structured, and predictable in its word choices. The problem is that plenty of human writing fits that profile. Formulaic school-essay structures, carefully edited prose, and — most troublingly — writing by students working in English as an additional language all trigger flags at elevated rates, a bias documented in published research. In a system where a flag can mean a misconduct hearing, even a small false positive rate is serious: run a few hundred essays a term through a detector with a modest error rate and you will accuse several innocent students a year, with no way of knowing which ones. An accusation of cheating, wrongly made and backed by a confident-looking percentage, damages trust in a way few classroom events can.

The evasion problem

Meanwhile, the students detectors are meant to catch evade them without difficulty. Light paraphrasing, a few inserted errors, prompting the AI to write less predictably, or running text through a rewriting tool all drop detection scores sharply — and students share these techniques within days of any detector being adopted. The result is the worst of both worlds: the tool is most likely to flag the conscientious student who wrote carefully, and least likely to flag the one who took ten minutes to launder machine output. A detector score is not evidence; treating it as evidence is the core mistake.

Design assessments where misuse does not pay

The durable response is not better detection but better design: assessing in ways that either make AI use irrelevant or make the student's own understanding impossible to fake. None of these ideas are new — they are old good practice that AI has made newly urgent.

• Oral defence: after submission, the student spends five minutes explaining their argument, justifying a choice, or answering one unexpected question. Understanding survives this; ghostwriting does not. Even applying it to a random sample of students changes the incentive for all of them.
• In-class writing: some assessment simply happens in the room, on paper or locked-down devices. It need not be everything — but if key grades include supervised components, the take-home work can safely become practice.
• Process portfolios: students submit the trail — plan, rough draft, revisions — not just the polished artefact. Document version history makes this nearly frictionless, and a plausible writing process is far harder to fabricate than a final essay.
• AI-open tasks: sometimes the honest move is to allow AI explicitly and raise the bar — critique the AI's draft, fact-check its claims, improve its argument. The assessed skill shifts to judgement, which is the skill the AI era actually demands.

The conversation beats the tribunal

When suspicion does arise, the productive path is pedagogical rather than forensic: sit down with the student and talk through the work. 'Walk me through how you approached this paragraph' reveals more in three minutes than any percentage score, and it does so without an accusation. Students who did the work demonstrate it readily; students who did not usually concede quickly when the gap is exposed gently. Either way the teacher learns something true — which is precisely what detectors, for all their confident numbers, cannot provide.