How we test every number we publish
Accuracy claims are easy to make and hard to check. Here is what we measure, the writing we test on, where Winston AI gets it wrong, and the full method so you can repeat it.
Evaluation 2026.3, updated October 2026. PDF, 28 pages.
Evaluation 2026.3 at a glance
Sample figures- accuracy on the core benchmark70,400 documents
- 99.4%
- of human-written essays wrongly flaggedAbout 6 in every 1,000
- 0.6%
- of AI text still caught after paraphrasing15,000 rewritten documents
- 94.1%
120,100 documents across 9 test sets, each reported on its own below, including the ones where we do worst.
How we measure
Three numbers describe how a detector behaves. Each answers a different question, and each can look good while the others look bad, so we publish all three.
Accuracy
How often the label is right: human for human writing, AI for AI writing.
Calculated as: Correct labels ÷ all documents
A useful summary, but an average can hide a weak spot in a smaller group. That's why we never report it alone.
False-positive rate
How often writing by a person gets flagged as AI.
Calculated as: Human documents flagged as AI ÷ all human documents
This is the mistake that hurts people. We tune for it first and accept missing some AI text to keep it low.
Adversarial robustness
How well detection holds up when someone tries to hide AI writing.
Calculated as: Rewritten AI documents still flagged ÷ all rewritten AI documents
Paraphrasers and humanizer tools are cheap and easy. A detector that only works on untouched output isn't much use.
What a 0.6% false-positive rate looks like
Picture 1,000 essays, all written by students. About six of them would be flagged as AI. That's six people who deserve a careful second look, not an accusation, which is why every result shows the sentences behind the score.
- Wrongly flagged: 6
- Correctly left alone: 994
Results by test set
Every set is scored with the same model and the same thresholds you get in the product. Nothing is tuned per test.
Sample figures, to be replaced with published results. They show the format of the report, not measured performance.
Swipe the table sideways to see every column.
| Test set | Documents | Accuracy | False positives | False negatives |
|---|---|---|---|---|
| Core benchmarkThe headline figures come from these sets. | ||||
| Student essaysWritten 2015–2021, before AI tools | 18,400 | 99.4% | 0.6% | Not applicable |
| Published articlesNews, magazine and blog posts | 12,000 | 99.5% | 0.5% | Not applicable |
| Current model outputs9 models, 40 genres of prompt | 40,000 | 99.3% | Not applicable | 0.7% |
| Stress testsHarder conditions, reported separately so they can't hide in an average. | ||||
| Paraphrased and humanizedAI text run through rewriting tools | 15,000 | 94.1% | Not applicable | 5.9% |
| Mixed human and AIScored sentence by sentence | 16,000 | 91.3% | 4.2% | 12.6% |
| Non-native English writersEssays by second-language students | 6,200 | 98.1% | 1.9% | Not applicable |
| Short texts40 to 150 words | 5,000 | 96.9% | 3.1% | Not applicable |
| Grammar-tool editedHuman essays polished by writing assistants | 4,000 | 98.8% | 1.2% | Not applicable |
| Technical writingAbstracts, manuals and documentation | 3,500 | 99.2% | 0.8% | Not applicable |
- False positives:
- human writing labeled as AI.
- False negatives:
- AI writing labeled as human.
- A dash:
- the set has no text of that kind, so the error can't happen.
Where honest writing gets flagged
Short, formulaic or tool-polished writing gives any detector less to go on. These are the groups where people are most often flagged by mistake.
We report them separately so they can't hide in an average, and the product warns you when a scan falls into one of them.
False-positive rate by type of human writing
Share of human-written documents labeled as AI. Lower is better. Sample figures.
- Short texts: 3.1% false positives, about 155 of 5,000 documents
- Non-native English writers: 1.9% false positives, about 118 of 6,200 documents
- Grammar-tool edited: 1.2% false positives, about 48 of 4,000 documents
- Technical writing: 0.8% false positives, about 28 of 3,500 documents
- Student essays: 0.6% false positives, about 110 of 18,400 documents
- Published articles: 0.5% false positives, about 60 of 12,000 documents
Show as a table
| Type of writing | False positives | Documents |
|---|---|---|
| Short texts | 3.1% | 5,000 |
| Non-native English writers | 1.9% | 6,200 |
| Grammar-tool edited | 1.2% | 4,000 |
| Technical writing | 0.8% | 3,500 |
| Student essays | 0.6% | 18,400 |
| Published articles | 0.5% | 12,000 |
Our test sets
A detector is only as trustworthy as the writing it's tested on. Ours mixes clean examples with the messy cases you see in real life.
Known human writing
Essays, articles and technical writing submitted or published before 2022, so none of it could have come from a chatbot. Includes second-language writers and short passages.
- 49,100 documents
- Written 2015–2021
Current model outputs
Text from nine models, including ChatGPT, Claude, Gemini and Llama, written to 40 kinds of prompt: essays, emails, product copy, cover letters and more.
- 40,000 documents
- New models added each quarter
Adversarial rewrites
AI text run through paraphrasers, humanizer tools and synonym swaps, prompted to sound casual or make mistakes, then lightly edited by people.
- 15,000 documents
- 12 rewriting tools
Mixed documents
Human drafts with AI paragraphs dropped in, and AI drafts reworked by people. Every sentence is labeled by hand, so we can score the highlights and not just the overall verdict.
- 16,000 documents
- Labeled by two reviewers
Ground rules
Four rules that keep the numbers honest.
Never used for training
No document in a test set is ever shown to the model while it learns.
Same settings as the product
The model version and thresholds match what you use when you scan.
Refreshed every quarter
New models and new rewriting tools are added as they appear.
Reported set by set
An overall average is always shown next to the parts that make it up.
Limitations
No detector is right every time, including ours. These are the cases where you should trust a result less, and what to do about it.
Never use a score as the only evidence.
A result is a reason to look closer, not proof. Before acting on it, read the highlighted sentences, compare them with the writer's earlier work and talk to them.
Short texts
Under about 150 words there are too few sentences to judge. Scores swing more, and in our tests false positives are around five times higher than on full essays.
What to do: Scan a longer sample, or treat the result as a weak signal. Short scans carry a warning in the report.
Heavily edited text
When a person reworks AI text line by line, or a writing assistant polishes human text, the boundary blurs. Expect mixed results, and expect some to be wrong in either direction.
What to do: Read which sentences are flagged instead of relying on the overall score.
Non-native English writers
Second-language writing can be more formulaic, which detectors can mistake for AI. We test for it directly, but the false-positive rate is still about three times our headline figure.
What to do: Weigh results for second-language writers with extra care and more context.
Languages other than English
We test all 14 supported languages, but our test sets are smaller outside English, so the margin of error is wider.
What to do: Check the language shown in the report matches the text.
Brand-new models
A model released last week may not be in our test sets yet. We add new models within weeks, but there is always a gap.
What to do: Expect accuracy on the newest models to improve after each quarterly update.
Read the full methodology
Everything on this page in detail, with enough information for another team to repeat the test and check whether they get the same results.
- Where every document came from
- How documents and sentences are labeled
- Model version and thresholds
- Results by model and by language
- Error analysis with real examples
- How to reproduce our numbers
PDF, 28 pages. Researchers can request the held-out test sets under a data agreement.
Check our work on your own writing
Paste something you wrote, then something a chatbot wrote, and compare the highlights. No account needed.