AI detector false positives: why honest writing gets flagged
A false positive is human-written text that a detector labels as AI-generated. Every detector produces some, including ours. This guide explains why, who is affected most, and how to read a score without doing harm.
What a false positive is
There are two ways a detector can be wrong. A false positive flags human writing as AI. A false negative lets AI writing through as human. False positives are the more serious error in education and employment, because they can land on someone who did nothing wrong.
Why they happen
Detectors measure resemblance to machine-generated patterns, and plenty of human writing resembles them.
- Predictable or formulaic text. Templates, lab reports, definitions and other structured genres are even and repetitive by design. OpenAI’s withdrawn classifier listed highly predictable text as something it could not classify reliably.
- A constrained vocabulary. Someone writing in a second language often uses common words and simple constructions. That resembles the low-variety pattern detectors look for.
- Short passages. With little text to measure, one unusual habit can swing the result.
- Heavy polish. Text edited until every sentence is tidy can lose the natural unevenness that detectors treat as human.
Who is affected most
A 2023 study in the journal Patterns tested several widely used GPT detectors and found that they consistently misclassified non-native English writing samples as AI-generated, while accurately identifying native writing. The authors suggested detectors may penalise writers with a more constrained linguistic range, and argued that using them in evaluative or educational settings raises ethical concerns, because they could unfairly disadvantage non-native English speakers.
Vendors have responded since. GPTZero, for example, says its detector is de-biased for ESL writers. Those are the vendors’ own claims, and we have not verified them. Litimus makes no such claim: it is a simple signal-based tool tuned for English, and it is not immune to this problem.
What the vendors say about their own error rates
Turnitin says it targets a false positive rate below 1%, acknowledges a small risk remains, and says its AI writing report should not be the sole basis for adverse action against a student. It also hides scores between 1% and 19% behind an asterisk to reduce false positives. Those are careful statements, and they still leave room for wrongly flagged students.
Why a small rate still matters
Suppose a detector wrongly flags one honest document in a hundred. Across 10,000 honest documents that is about 100 people. In a class of thirty honest students, the chance that at least one is wrongly flagged on a single assignment is roughly one in four, and it grows with every assignment. This is an illustration of the arithmetic, not a measured rate for any product. It is why a flag should start a conversation and not end one.
How Litimus tries to limit the harm
- Middle scores, from 35 to 65, are labelled mixed / unclear rather than pushed to one side.
- The report shows the readings and quotes the phrases behind the score, so a reader can judge whether they mean anything in context.
- Passages under 25 words are flagged as too short for a stable read.
- Our Terms of Service say a score is a probability and not proof, and ask you not to use it as the sole basis for a penalty.
None of this removes false positives. It only makes them easier to spot.
How to read a borderline score
Treat it as inconclusive. Look at which signals drove it: a handful of stock transitions is a different story from uniformly even sentences. Compare against the writer’s other work. Ask about their process and ask to see drafts or version history. If the writer is a non-native English speaker, weigh the score even lower. And if you are the writer, read what to do if you are falsely accused.
Sources
- Liang et al., “GPT detectors are biased against non-native English writers” (Patterns, 2023)
- Turnitin: Understanding false positives within our AI writing detection capabilities
- Turnitin Guides: Using the AI Writing Report
- OpenAI: New AI classifier for indicating AI-written text (with the July 2023 discontinuation notice)
- GPTZero