Are AI detectors accurate? What the tests found and what to check instead

TL;DR

• No. AI detectors make a statistical guess, and independent tests keep finding them too unreliable to prove who wrote a piece of text.
• OpenAI pulled its own AI classifier in July 2023 after it caught 26% of AI-written text and wrongly flagged 9% of human writing.
• Seven popular detectors labelled 61% of essays by non-native English writers as AI-generated, while scoring essays by US-born eighth-graders almost perfectly.
• Small tweaks fool them. Changing the prompt cut detection of AI-written essays from 100% to 13%.
• For marketers, a detector score tells you very little. Check facts, voice, and originality instead, because that is what Google and your readers judge.

An AI detector works like a smoke alarm that can't tell burnt toast from a house fire. It shrieks at both, and now and then it stays silent while the kitchen burns. You learn something when it beeps, just never enough to call the fire brigade.

Below you'll find how AI detectors work, what the independent tests found, why they flag human writing, how they compare with the new watermarks from OpenAI and Anthropic, and what a small content team should check instead of a score.

Are AI detectors accurate?

No. Across 14 tools, including Turnitin and PlagiarismCheck, the detectors proved neither accurate nor reliable, and they leaned towards calling AI text human-written. They miss AI text, flag human text, and disagree with each other.

OpenAI, the company behind ChatGPT, couldn't make detection work either. Its own classifier identified 26% of AI-written text and labelled human writing as AI 9% of the time, and OpenAI withdrew it on 20 July 2023 for its low rate of accuracy. If the people with the training data gave up, a free tool with a big green percentage won't do better.

Turnitin still publishes confident numbers, and puts its sentence-level false positive rate at around 4%, so four in every 100 sentences it highlights as AI could be human writing. Read that as the vendor's own figure.

How do AI detectors work?

AI detectors score how predictable your writing is. Language models pick likely next words, so text full of safe word choices and evenly sized sentences looks machine-made to a detector. The main metric is perplexity, which tracks the sophistication of the writing.

The output is a probability dressed up as a verdict. A detector has no idea who wrote the text. It only knows how closely the text matches patterns in its training data, which is why the same paragraph can score low in one tool and high in the next.

Why do AI detectors flag human writing?

Detectors flag plain, careful writing, because that style is predictable. Non-native English writers get hit hardest. Seven detectors classified 61.22% of TOEFL essays by non-native students as AI-generated, while staying near-perfect on essays from US-born eighth-graders. At least one detector flagged 89 of the 91 TOEFL essays.

Marketing copy runs into the same wall. A product page edited to house style, a technical explainer, or a tidy LinkedIn post can all trip a detector, because good editing strips out the quirks a detector reads as human.

Can AI detectors catch edited AI text?

Barely. Simple changes to the prompt cut detection of AI-written college essays from 100% to 13%. Paraphrasing tools and manual edits also made the 14 tested tools perform significantly worse.

So detectors fail in both directions. They punish careful human writers and wave through anyone willing to spend five minutes rewording. For a tool sold as proof, that is the worst possible combination.

AI detector accuracy at a glance

AI detectors vs watermarks: what's the difference?

A detector guesses from the outside. A watermark gets added by the model provider at the moment of writing. Anthropic added a watermark to Claude's text worldwide, and OpenAI is rolling out a ChatGPT watermark in the EU. Both hide a statistical pattern in word choice that only the provider's key can read.

Watermarks are the industry's response to detectors failing, and they have limits too. Heavy editing weakens them, and a missing watermark proves nothing about who wrote the text. Neither of them tell you whether the piece is any good.

We see the worry in our own Search Console. A steady stream of searches lands on contengi.com from people asking how to strip a Claude watermark. Hiding where a draft came from leaves everything else about it exactly as it was, generic sections included.

Does Google use AI detectors to judge content?

Google judges content on accuracy, quality, and relevance, and it treats generating many pages without adding value for users as scaled content abuse. Nothing in its guidance ties rankings to a detector score.

So put the effort into the page itself. A page full of first-hand detail can rank even if a detector flags it. A page that passes every detector and repeats what nine other pages say has nothing to rank on, and that is what makes AI content hurt SEO.

It's also why Contengi doesn't score drafts against a detector. Every draft starts from the brand's anti-slop standard and hard rules, Claude edits its own draft, plain code scans for exact rule breaks, and a second agent checks for the patterns a word list misses.

What to check instead of an AI detector score

1. Check every fact against a source. Generative models predict likely words, so names, numbers, and dates need a human check before anything goes live, titles and meta descriptions included.
2. Read it out loud for voice. If the paragraph could sit on a competitor's site unchanged, rewrite it against your tone rules. A written brand voice that AI can follow does most of this work before the draft exists.
3. Run a slop check. The free AI slop cleaner strips hidden characters and highlights the tells readers spot, like stock phrases, fake-suspense questions, and sentences that all run the same length.
4. Add something only you have. A customer call, your own numbers, or an opinion you'd defend in a client meeting. That's the core of a non-commodity content strategy.
5. Edit for substance. Cutting generic sections and rewriting the opening changes far more than any paraphraser, and it's the fastest way to make AI writing sound human.
6. Record who reviewed it. A name and a date on each published piece answers any question about editorial responsibility, which a detector score never will.

Pick your last three published posts and run them through this list this week. You'll learn more from that hour than from any percentage a detector hands you.

Frequently asked questions

Are AI detectors accurate?

No. Independent tests find they miss edited AI text and flag human writing, especially writing by non-native English speakers. OpenAI withdrew its own detector in 2023 because of its low accuracy.

Can an AI detector be wrong about human writing?

Yes, and often. Formal, plain, or carefully edited writing scores as predictable, which detectors read as AI. Turnitin puts its own sentence-level false positive rate at around 4%.

Why do AI detectors give different results for the same text?

Each detector is trained on different data and uses its own thresholds. In one test of seven detectors, at least one flagged 89 of 91 human-written essays, while all seven agreed on just 18.

Is a 20% AI detection score bad?

A score is a probability, so 20% means the tool spotted some patterns it associates with AI. With false positives this common, a low or middling score proves nothing either way, so check the facts and the voice instead.

Does Google penalise content that AI detectors flag?

Google's guidance focuses on accuracy, quality, and whether a page adds value for users, and it doesn't tie rankings to detector scores. The risk sits with mass-produced pages that add nothing new.