AI Text Detection in arXiv Papers: ReText.AI Study of 11,848 Scientific Preprints
Large language models have changed the way people draft, edit, translate, and structure written work. In academic and scientific writing, this change is especially difficult to measure: researchers may use AI tools for language polishing, outline generation, translation support, code explanations, literature summaries, or full paragraph drafting.
That makes a simple question surprisingly hard to answer:
How much has scientific writing changed since public LLMs became widely available?
To explore this, ReText.AI analyzed 11,848 English-language arXiv papers submitted between 2015 and 2026. The study used an AI text detector to measure AI-like writing signals across more than 310 million characters extracted from LaTeX source files.
The goal was not to identify whether any individual paper was written by AI. Instead, the study looks at aggregate, corpus-level trends: how AI-like writing patterns changed over time, whether the shift remains visible after controlling for arXiv fields, and where in papers AI-like signals appear most strongly.
Important note: In this article, “AI-like” does not mean proven AI authorship. It means that a text segment was classified by the detector as resembling AI-generated or substantially AI-assisted writing.
Key findings
The study found several notable shifts in the arXiv sample:
- The article-level mean AI-like share increased from 36.2% in 2023 to 51.8% in the 2026 year-to-date sample.
- The share of papers with AI-like score ≥50% more than doubled, from 24.8% in 2023 to 52.3% in 2026.
- The sharpest visible break appeared in 2024, after a relatively stable 2015–2023 baseline.
- The trend remains visible after field normalization, meaning it is not explained only by changes in the mix of arXiv fields.
- AI-like signals were strongest in the opening 20% of papers.
These findings suggest a broad change in the language of scientific preprints after the widespread adoption of public LLM tools.
What the AI detector found
From 2015 to 2023, the AI-like share in the sample stayed relatively stable. The article-level mean moved from 32.7% in 2015 to 36.2% in 2023, with only moderate year-to-year variation.
The trend changed sharply after that. In 2024, the mean AI-like share rose to 44.6%. In 2025, it reached 49.4%. In the 2026 year-to-date sample, it reached 51.8%.
The median followed the same pattern. It was 34.0% in 2023, then rose to 43.0% in 2024, 47.4% in 2025, and 51.6% in 2026.
The fact that both the mean and the median rise is important. A rise in the mean alone could be driven by a small number of unusually high-scoring papers. A rise in both metrics suggests a broader shift across the corpus.
This does not prove that individual papers were written by AI. It shows that the language of the analyzed arXiv corpus became more similar to AI-generated or AI-assisted writing patterns after 2023.
Papers with majority AI-like text more than doubled
Average values are useful, but they do not show how many papers crossed a high AI-like threshold. For that reason, ReText.AI also measured the share of papers where at least half of the analyzed text was classified as AI-like.
This metric changed sharply:
- 2023: 24.8% of papers had AI-like score ≥50%
- 2024: 40.1%
- 2025: 47.1%
- 2026: 52.3%
In other words, in the analyzed sample, papers with majority AI-like text increased from roughly one quarter of the corpus in 2023 to just over half of the corpus in 2026.
This is one of the clearest signals in the study. It shows not only that the average score rose, but that a much larger portion of the corpus moved into a higher AI-like range.
Does the trend come from one arXiv field?
arXiv is not a single homogeneous corpus. It includes papers from multiple fields, including Computer Science, Mathematics, Physics, Statistics, Electrical Engineering and Systems Science, Quantitative Biology, Quantitative Finance, and Economics.
That creates a methodological question:
Could the overall increase be explained by a change in field composition?
For example, if one year contained more Computer Science papers and fewer Mathematics papers, the overall trend might change even if writing patterns inside each field stayed the same.
To address this, ReText.AI compared the raw trend with a field-normalized trend. Field normalization uses fixed arXiv field weights, so the year-to-year trend is not driven only by changes in the mix of fields.
The result: the field-normalized line remains close to the raw line and shows the same sharp rise after 2023.
This makes the finding stronger. The rise in AI-like signals is not simply an artifact of the sample containing more papers from fields that already had higher AI-like scores.
AI-like share rose across multiple scientific fields
The field-level comparison also shows that the increase was not limited to one area of research.
ReText.AI compared the article-level mean AI-like share in two periods:
- 2022–2023: the baseline / transition period
- 2025–2026: the later period after public LLM adoption became widespread
Every analyzed arXiv field showed a higher AI-like share in the later period.
The highest late-period values appeared in:
- Quantitative Finance
- Electrical Engineering and Systems Science
- Computer Science
- Physics
- Statistics
The field differences should be interpreted carefully. A higher score in a field does not necessarily mean more AI-generated writing. Some fields may use more standardized phrasing, formulaic abstracts, structured contribution statements, or repetitive methodological language. But the cross-field increase indicates that the shift is broader than a single category.
AI-like signals are strongest near the opening of papers
ReText.AI also examined where AI-like signals appear inside papers.
Instead of relying on section titles such as “Introduction” or “Conclusion,” the study used paragraph position:
- Opening 20%
- Middle 60%
- Closing 20%
This avoids inconsistencies in how arXiv papers are structured and labeled.
The strongest AI-like share appeared in the opening 20% of papers:
- Opening 20%: 52.2%
- Middle 60%: 32.6%
- Closing 20%: 36.1%
One possible explanation is genre. Opening sections often contain standardized language around motivation, context, research gaps, contribution statements, and paper structure. These are exactly the kinds of passages where LLM-assisted drafting or polishing may be more common — and where formal academic writing may also resemble AI-like style.
The middle of a paper often contains more equations, data, experiments, citations, technical details, or field-specific argumentation. Those elements may produce lower AI-like scores or may be excluded from the analyzed prose if they appear as formulas, tables, or non-prose blocks.
How the AI text detector worked
The analysis was conducted using ReText.AI’s AI text detector, which classifies eligible paragraphs as human-like or AI-like based on linguistic and statistical patterns. The detector was used to measure aggregate writing-pattern signals across the corpus, not to judge individual papers or prove AI authorship.
The study used a paragraph-level AI text detection pipeline. Each eligible paragraph was classified independently by ReText.AI’s detector.
The workflow had four main stages.
1. Data collection
The corpus included English-language arXiv papers from 2015 to 2026, sampled from a broad set of fields:
- Computer Science
- Mathematics
- Physics
- Statistics
- Electrical Engineering and Systems Science
- Quantitative Biology
- Quantitative Finance
- Economics
The sample included roughly 1,000 papers per year.
2. Text extraction
Text was extracted from LaTeX source files, not from PDF documents. This reduced noise from PDF parsing and made it easier to exclude non-prose elements.
The extraction process removed:
- LaTeX preambles
- formulas
- figures
- tables
- code listings
- bibliography sections
- templates
- short fragments
- non-prose blocks
Only connected English prose was kept for analysis.
Paragraphs shorter than 300 characters were excluded, and papers with fewer than five eligible paragraphs were removed from the sample.
3. Paragraph classification
Each paragraph was analyzed by an LLM-based detector using Qwen3.5-9B with LoRA adaptation. The detector assigned a binary label indicating whether the paragraph resembled human-written or AI-like writing.
4. Aggregation
For each paper, ReText.AI calculated an AI-like share:
AI-like share = characters in AI-like paragraphs ÷ all analyzed characters
The study then aggregated the results by year, field, and paragraph position.
The report uses several metrics:
- Article-level mean: each paper counts as one observation, then the yearly average is calculated.
- Article-level median: the median paper-level AI-like share for each year.
- Char-weighted share: longer papers carry more weight because the metric is weighted by analyzed character count.
- Share ≥50%: the percentage of papers where at least half of the analyzed text was classified as AI-like.
- Field-normalized trend: a trend calculated with fixed arXiv field weights.
What “AI-like” means — and what it does not mean
AI detection is not the same as authorship verification.
An AI detector evaluates linguistic and statistical patterns. It can estimate whether a text resembles AI-generated or AI-assisted writing, but it cannot prove how the text was created.
That distinction matters. A high AI-like score may reflect several different scenarios:
- text generated by an LLM;
- AI-assisted editing;
- translation or rewriting support;
- heavy paraphrasing;
- standardized academic phrasing;
- templated section structure;
- formal language that resembles LLM output.
This is why ReText.AI interprets the arXiv results as aggregate writing-pattern signals, not as evidence against individual authors or papers.
For individual text checks, users should treat AI detection as one signal among several. It can be useful for reviewing drafts, identifying overly generic passages, or checking whether a text looks strongly AI-like before submission. It should not be used as a standalone verdict.
Why the baseline matters
One of the most important findings is that AI-like signals existed before public LLMs became widely available.
From 2015 to 2023, the detector consistently found a baseline level of AI-like language in arXiv papers. This does not mean those papers were generated by AI. It means that formal scientific writing can sometimes resemble the patterns that AI detectors associate with generated text.
This baseline can come from:
- standardized academic language;
- repeated rhetorical structures;
- formulaic abstracts and introductions;
- conventional phrases used across papers;
- translation into English;
- field-specific writing conventions.
That is why the study focuses on change over time. The key signal is not the existence of AI-like text before 2022. The key signal is the sharp upward shift that appears after the mass adoption of public LLM tools.
What the findings suggest about AI-assisted scientific writing
The results point to a broader change in scientific writing habits.
LLMs are likely being used in several different ways:
- drafting abstracts and introductions;
- improving English style;
- rewriting dense technical passages;
- summarizing related work;
- generating paper structure;
- polishing conclusions;
- converting rough notes into formal prose.
Some of these uses may be acceptable or even helpful depending on institutional rules, journal policies, and disclosure requirements. Others may raise concerns around transparency, authorship, originality, and accountability.
The study does not determine which use case applies to any specific paper. It only shows that, at corpus level, the language of arXiv papers has become more AI-like since 2024.
How researchers can use AI tools more responsibly
AI-assisted writing is not automatically problematic. The risk comes from using AI tools without review, without domain expertise, or without understanding how generic generated language can weaken the quality of a paper.
For researchers, students, and editors, a safer workflow is:
- Use AI tools for drafting support, not final authorship.
- Review every claim, citation, and technical statement manually.
- Replace generic language with field-specific reasoning.
- Add concrete details, methods, results, and limitations.
- Check whether the final text sounds repetitive or overly templated.
- Use an AI detector as a diagnostic tool, not as a verdict.
If a draft appears too generic or too AI-like, rewriting should not be about “hiding AI.” It should be about making the text clearer, more specific, and more genuinely useful.
The best use case is responsible editing: removing generic phrasing, improving readability, and making the text sound less mechanical without changing the substance.
Limitations of the study
This study has several important limitations.
First, AI detection does not prove authorship. The results should not be used to accuse individual authors or evaluate individual papers.
Second, the corpus is limited to English-language arXiv papers. The findings should not be generalized to all scientific publishing, peer-reviewed journals, non-English papers, or other academic databases.
Third, the 2026 result is preliminary and based on the year-to-date sample.
Fourth, the detector cannot distinguish precisely between full AI generation, AI-assisted editing, translation, paraphrasing, or support with isolated passages.
Fifth, formal academic writing can produce background AI-like detections even before the modern LLM period.
For these reasons, the study is best understood as a trend analysis. It shows how AI-like writing signals changed across a large corpus of scientific preprints over time.
Conclusion
The ReText.AI arXiv study shows a clear shift in scientific writing patterns after 2023.
The article-level mean AI-like share increased from 36.2% in 2023 to 51.8% in the 2026 year-to-date sample. The share of papers with AI-like score ≥50% rose from 24.8% to 52.3% over the same period.
The increase remains visible after controlling for arXiv field composition, and it appears across multiple fields. AI-like signals are strongest in the opening parts of papers, where scientific writing often relies on standardized framing, motivation, and contribution language.
These results do not prove that individual papers were written by AI. They show that public LLMs are likely reshaping the style and structure of scientific writing at scale.
As AI-assisted writing becomes more common, researchers, universities, publishers, and tool providers need more nuanced ways to interpret AI detection results. The future of AI in academic writing is not just about detecting AI-generated text. It is about understanding how AI changes writing — and how to use it responsibly.
FAQ
What is AI text detection?
AI text detection is the process of analyzing linguistic and statistical features to estimate whether a text resembles AI-generated or AI-assisted writing. It does not prove authorship and should not be used as the only basis for decisions about a text.
What does AI-like score mean?
An AI-like score estimates how strongly a text resembles patterns commonly found in AI-generated or substantially AI-assisted writing. In this study, AI-like share was calculated as the proportion of analyzed characters contained in paragraphs classified as AI-like.
Can an AI detector prove that a paper was written by AI?
No. An AI detector cannot verify how a document was created. It can only provide a signal based on textual patterns. Formal academic writing, translation, templated phrasing, and editing support can all affect the result.
Why did ReText.AI analyze arXiv papers?
arXiv provides a large corpus of scientific preprints across multiple fields and years. Because many papers are available as LaTeX source files, the text can be extracted more cleanly than from PDFs.
Why did the study use LaTeX source files instead of PDFs?
LaTeX source files make it easier to remove formulas, tables, bibliography, code listings, and other non-prose elements. This helps focus the analysis on connected English prose rather than PDF parsing artifacts.
What changed after 2023?
The study found a sharp increase in AI-like writing signals beginning in 2024. Both the article-level mean and median increased, and the share of papers with AI-like score ≥50% rose substantially.