Why AI detection tools don’t work as evidence, and what to do instead

AI detectors promise to tell human writing from machine writing. The newest ones have improved, but the research is clear that a detector score cannot show how a real text was written. Here is the evidence, what librarians, integrity researchers and universities now recommend, and what the European Commission’s guidelines ask for instead.

Updated 2026-10-09

The short answer

AI detectors cannot tell you how a real piece of research writing was produced. The best current tools are good at one narrow task: spotting text that an AI wrote from start to finish and nobody edited. Real writing is rarely like that. Researchers draft, edit, polish with AI, rewrite and add their own data, and on that kind of mixed text detectors are unreliable, easy to fool and sometimes wrong about fully human writing.

That is why OpenAI withdrew its own detector, why universities are switching detection off, and why the EU guidelines on generative AI in research focus on transparency and responsibility instead. AI is a tool. Nothing about using it is shameful. What matters is that the work is good, honest about its methods and yours.

Why early AI text was easy to detect

Early language models wrote in a recognisable way. When OpenAI released GPT-2 in 2019, it also released a detector with detection rates of about 95%, and even then said this was not accurate enough to use on its own. Text from those models was statistically predictable: each word tended to be the most likely next word. Detectors learned to spot that low "perplexity".

Modern models write with far more variety, follow style instructions and can imitate a particular author. And careful human writing, especially by people writing in a second language, often has exactly the predictable, simple features detectors associate with AI. That is the root of most false accusations.

The evidence, from 2023 to 2026

These findings come from the original sources listed at the end of this guide.

  • OpenAI AI Classifier (2023): at launch it correctly identified only 26% of AI-written text and wrongly flagged 9% of human text. OpenAI withdrew it on 20 July 2023, citing its low rate of accuracy.
  • Liang and colleagues, Patterns (2023): seven popular detectors flagged on average 61% of essays written by non-native English speakers as AI-generated, while classifying essays by US students correctly.
  • Weber-Wulff and colleagues, International Journal for Educational Integrity (2023): 14 tools, including Turnitin, were tested. None reached 80% accuracy, and the authors concluded the tools are "neither accurate nor reliable".
  • Perkins and colleagues, International Journal of Educational Technology in Higher Education (2024): seven popular detectors had a mean accuracy of 39.5% on AI text. Simple editing techniques lowered it to about 22%, and human-written samples were classified correctly only 67% of the time.
  • Sadasivan and colleagues, University of Maryland (2023, updated 2025): recursive paraphrasing sharply reduced detection rates for every kind of detector tested, including watermark-based ones.
  • Cheng and colleagues, NeurIPS 2025: paraphrasing guided by a detector cut detection rates by about 88% on average across detectors.
  • Saha and Feizi, ACL Findings 2025: detectors often flagged human text that AI had only lightly polished as AI-generated.

Have newer detectors fixed the problem?

Partly, and it is fair to say so. A 2025 University of Chicago Booth study by Jabarian and Imas found that the commercial detector Pangram had near-zero false positive and false negative rates, even on short passages and text run through humanizer tools. Other detectors did less well, especially against humanizers. In July 2026 Epoch AI found that Pangram and GPTZero flagged none of 495 human passages written before 2022.

The same studies show the limits. They tested text that was either fully human or fully AI, not the mixed, edited text of real manuscripts. Epoch AI found that when AI imitated the style of real authors, detectors missed 10% to 18% of it, and for scientific writing about a quarter. Humanizer tools still defeat some detectors. And as Bassett and colleagues (2026) point out, in a real case nobody knows the true origin of the text, so a detector’s accuracy on that text cannot be checked.

There is also simple arithmetic. Turnitin reports a sentence-level false positive rate of around 4%. Vanderbilt University calculated that even a 1% rate would have wrongly flagged about 750 of the 75,000 papers it submitted in one year. Washington State University made the same calculation for about 148,000 assessments when it cancelled its detection contract in 2026.

What librarians, integrity researchers and universities say

Sarah Elaine Eaton, an academic integrity researcher at the University of Calgary, has written that "the detection paradigm has failed on its own terms" and that detectors can cause real harm, especially for multilingual students. MIT Sloan’s teaching and learning team publishes guidance titled "AI detectors don’t work" and recommends asking for a short statement of how the work was done instead. Jisc, the UK’s national digital body for higher education, advises that decisions should never be based solely on AI detection.

Even Turnitin says its AI score should not be the sole basis for action against a student, and it no longer shows scores between 1% and 19% because of a higher rate of false positives.

Universities have acted. Vanderbilt disabled Turnitin’s AI detector in 2023, and the University of Glasgow and the University of Limerick opted out. Curtin University switched AI detection off from January 2026, Washington State University cancelled its contract in February 2026, and the University of the Free State in South Africa discontinued AI detection from July 2026, citing growing evidence that the tools cannot consistently tell human and machine writing apart.

What about watermarks and the EU AI Act?

A different approach is to mark AI output at the source. Google’s SynthID Text, described in Nature in 2024, hides a statistical watermark in generated text. Its authors are open about the limits: it needs the AI provider to cooperate, it is weakened by edits and paraphrasing, and it is not a complete solution.

The EU AI Act requires providers of generative AI to mark synthetic content in a machine-readable way, with the transparency rules applying from August 2026, and a Code of Practice on the transparency of AI-generated content was finalised in June 2026. This helps identify media such as images and deepfakes. For text that a researcher has edited and rewritten, a watermark will not say who did the thinking.

What the EU guidelines ask for instead

The European Commission’s "Living guidelines on the responsible use of generative AI in research" (third version, May 2026) do not ask researchers to prove their text is human. They build on the European Code of Conduct for Research Integrity: reliability, honesty, respect and accountability. For researchers they recommend:

  • Remain ultimately responsible for scientific output. AI systems are neither authors nor co-authors.
  • Use generative AI transparently. Name the tools you used substantially and describe how. Basic editorial help is not substantial use; a literature review, data analysis or hypotheses can be.
  • Protect privacy, confidentiality and intellectual property. Do not upload unpublished or personal data to tools that may reuse it.
  • Respect the law, including copyright and data protection, and cite others properly.
  • Keep learning how to use AI tools well.
  • Avoid substantial AI use in sensitive assessments such as peer review and evaluating funding proposals.

Disclosure should be safe

The guidelines also ask research organisations to create an atmosphere of trust in which researchers can disclose AI use without fear, and they cite research showing that disclosed AI help can lead to lower ratings. Publishers agree on the direction. Springer Nature’s policy says plainly that transparency creates trust, and Elsevier, Wiley and others ask for a short statement rather than proof.

AI is a tool, like a statistics package, a reference manager or a spell checker. It can be used well or badly. The questions that matter are the same as always: is the work accurate, original in its contribution, honest about its methods, and does the author stand behind it?

How to use AI in your writing with confidence

  • Start from your own sources and ideas. Use AI to structure, draft and edit, not to decide what you think.
  • Check every claim and every reference. AI can invent citations or summarise papers wrongly, and you are responsible for catching it.
  • Make it your own voice. Rewrite, cut, add your data and examples, and remove anything you would not say.
  • Keep your drafts and version history. A record of how a text developed is far better evidence of your work than any detector score.
  • Disclose substantial use, following your journal or university’s rules.
  • Never paste participants’ personal data into a tool your institution has not approved.

How Kahubi supports honest AI use

Kahubi is built around the same principles. The agent writes from the papers in your library, cites them, and marks [CITATION NEEDED] when your sources do not support a claim instead of inventing a reference. It learns your style from your own published papers, so drafts start in your voice. Every change, yours or the AI’s, is saved in the version history, which gives you a clear record of how a text was made. Data is processed in the EU and never used for training.

Sources

  1. OpenAI (2023). New AI classifier for indicating AI-written text (withdrawn 20 July 2023).
  2. OpenAI (2019). GPT-2: 1.5B release.
  3. Liang W, Yuksekgonul M, Mao Y, Wu E, Zou J (2023). GPT detectors are biased against non-native English writers. Patterns 4(7).
  4. Weber-Wulff D, et al. (2023). Testing of detection tools for AI-generated text. International Journal for Educational Integrity 19:26.
  5. Perkins M, et al. (2024). Simple techniques to bypass GenAI text detectors: implications for inclusive education. International Journal of Educational Technology in Higher Education 21:53.
  6. Sadasivan VS, et al. (2023, revised 2025). Can AI-generated text be reliably detected? Transactions on Machine Learning Research.
  7. Cheng Y, Sadasivan VS, Saberi M, Saha S, Feizi S (2025). Adversarial paraphrasing: a universal attack for humanizing AI-generated text. NeurIPS 2025.
  8. Saha S, Feizi S (2025). Almost AI, almost human: the challenge of detecting AI-polished writing. Findings of ACL 2025.
  9. Jabarian B, Imas A (2025). Artificial writing and automated detection. NBER Working Paper 34223.
  10. Lee J (2026). AI detectors rarely flag human writing, but sometimes miss AI text imitating real authors. Epoch AI.
  11. Bassett M, et al. (2026). Heads we win, tails you lose: AI detectors in education. Journal of Higher Education Policy and Management.
  12. Turnitin (2023). Understanding the false positive rate for sentences of our AI writing detection capability.
  13. Turnitin. Using the AI Writing Report.
  14. Vanderbilt University (2023). Guidance on AI detection and why we’re disabling Turnitin’s AI detector.
  15. Washington State University, Office of the Provost (2026). Cancellation of Turnitin AI detection software.
  16. Curtin University (2025). Update on Turnitin AI detection tool.
  17. University of the Free State (2026). UFS shifts academic integrity approach in AI era.
  18. University of Limerick (2024). Turnitin and AI detection.
  19. University of Glasgow (2023). AI software update.
  20. Eaton SE (2026). Editorial. Journal of Educational Thought 59(1).
  21. University of Calgary (2026). Sarah Eaton: GenAI discussions offer opportunity to build ethical, equitable and inclusive learning.
  22. MIT Sloan Teaching & Learning Technologies. AI detectors don’t work. Here’s what to do instead.
  23. Jisc National Centre for AI (2025). AI detection and assessment: an update for 2025.
  24. Dathathri S, et al. (2024). Scalable watermarking for identifying large language model outputs. Nature 634:818-823.
  25. European Commission. Code of Practice on transparency of AI-generated content.
  26. Springer Nature. Artificial intelligence (AI) editorial policy.
  27. European Commission (2026). Living guidelines on the responsible use of generative AI in research, third version, May 2026.

Last updated 2026-10-09.

Frequently asked questions

Do AI detectors work?
The best current detectors are good at recognising text that AI wrote entirely and nobody edited. They are unreliable on real, mixed writing that has been drafted, edited and polished, they can be defeated by paraphrasing tools, and they sometimes flag human writing. A detector score should never be the only evidence of misconduct.
Why did OpenAI shut down its AI detector?
OpenAI withdrew its AI Classifier on 20 July 2023 because of its low rate of accuracy. At launch it correctly identified only 26% of AI-written text and wrongly flagged 9% of human-written text.
Are AI detectors biased against non-native English speakers?
A 2023 Stanford study in Patterns found that seven detectors flagged on average 61% of essays by non-native English speakers as AI-generated. Integrity researchers continue to warn that multilingual writers are at higher risk of false accusations.
Is Pangram or GPTZero accurate?
Independent tests in 2025 and 2026 found Pangram very accurate on fully human or fully AI text, and GPTZero accurate except against some humanizer tools. They missed more AI text when it imitated real authors or was scientific writing, and none of the tests covered human-edited AI drafts, which is how most researchers actually use AI.
What do the EU guidelines say about using AI in research?
The European Commission’s living guidelines ask researchers to stay responsible for their output, use AI transparently and disclose substantial use, protect privacy and confidentiality, respect the law, keep learning, and avoid substantial AI use in peer review and funding evaluation. They also ask organisations to make disclosure safe.
What should I do if I am falsely accused because of an AI detector?
Ask what evidence the accusation is based on. Point to the research on detector reliability and to the detector vendor’s own advice that a score is not proof. Share your drafts, notes, sources and version history to show how the work developed. Many universities no longer accept a detector score as evidence on its own.

Related

Write with AI, in your own voice

Drafts grounded in your sources, honest citations, full version history, processed in the EU. Free plan included.