How we test: inside our 2026 benchmark
Twenty thousand documents, nine model families and a strict rule about what counts as human. Exactly how we measure our detector, and where it falls short.
Maya Okafor
Head of research · Sep 11, 2026 · 9 min read
Benchmarks are easy to game. Pick easy samples, test against old models, and any detector looks great. We'd rather publish a benchmark that's hard on us, so here's how ours is built.
What's in the set
The 2026 set contains 20,000 documents split evenly between human and AI writing. Human samples come from consenting contributors and from sources written before generative models were widely available, so we can be confident they weren't AI-assisted.
- Student essays from secondary school through graduate level.
- Journalism, blog posts and marketing copy.
- Technical writing, from documentation to lab reports.
- Fiction, personal essays and transcribed speech.
- AI samples from nine model families, prompted the way real people prompt them.
How we score
Every document is scanned once with default settings. We report accuracy, false positive rate and recall per model, and we break results down by length and genre. Nothing is removed after the fact for being inconvenient.
If a benchmark never embarrasses you, it isn't testing much.
Where we still fall short
Heavily edited AI text remains the hardest case. Once a person has rewritten most sentences, the result is genuinely mixed, and our report says so. Very short text and poetry are also less reliable. We list these limits in the product, next to the score, not in a footnote.
The full dataset description and per-category results are on the research page. If you're evaluating detectors for a school or newsroom, we're happy to share the test protocol so you can run it yourself.
Written by
Maya Okafor, Head of research
Maya leads the team that builds and tests our detection models. Before that she spent eight years in computational linguistics.