Pew's 35% AI-authorship figure is now a corpus engineering problem

Pew Research Center published a study on August 20th reporting signs of AI writing or substantial editing on 35% of webpages that carry publication dates after ChatGPT's November 30th, 2022 launch ^[1][2]. For anyone assembling a retrieval corpus or a fine-tuning set out of crawled text, that moves provenance from a philosophical worry to a filter someone on your team has to write.

The measurement is specific. Samuel Bestvater, a senior data scientist at Pew and the study's lead researcher, ran 490,000 English-language pages collected by Common Crawl, drawing 10,000 pages from each of 49 snapshots created between January 2021 and July 2026 ^[3][4]. Those two figures reconcile exactly ^[8].

The denominator is narrower than "the web." Only about 10% to 15% of sampled pages carried publication dates in their HTML, which is how Pew sorted them into pre- and post-ChatGPT buckets ^[5]. The 35% therefore describes roughly one in seven to one in ten of the pages actually sampled ^[9]. Paywalled and login-gated material is likely underrepresented, since Common Crawl collects publicly accessible pages ^[6]. Across the entire July 2026 sample, including pages that predate widely available generative models, 10% showed significant signs of AI authorship ^[7]; the dated post-ChatGPT rate is about three and a half times that ^[11].

The instrument matters more than the headline. Pew scored body text with Open Pangram's editlens_Llama-3.2-3B model on a zero-to-one scale and treated 0.2 or higher as meaningful evidence ^[10]. That threshold captures pages a human drafted and a model substantially revised, so this is not a count of pure prompt output ^[12]. Pew cross-checked against the commercial Pangram 3.3 detector on 62,370 pages from seven crawls and found 96% agreement, while cautioning that page-level judgments remain probabilistic ^[13] - the two systems still split on roughly 2,500 pages ^[21]. About 1% of pre-ChatGPT pages came back AI-authored, which Pew read as a false-positive floor on older human writing ^[14]; net that out and the post-ChatGPT signal is still around 34 points ^[22]. Pangram, which released Open Pangram in March 2026 as a source-available research tool, explicitly warns against using the open models to enforce AI policies in schools or workplaces ^[15][16].

The useful part for pipeline work is that contamination is not uniform. In Pew's 2026 samples, roughly one in 10 .com pages showed signs of AI authorship, against 4.6% for .org and about 1% for both .edu and .gov ^[17] - an order of magnitude between commercial and government domains ^[23]. Top-level domain is a crude, nearly free first-pass weight. It also tells you the cost: .com is where product documentation, pricing pages and support content live.

Resist the urge to build a regex out of the stylometry. Compared with 2023 pages, em dashes appeared about twice as often, Oxford comma use rose 63%, a basket of AI-favored words more than doubled, and negative parallelism nearly tripled from an uncommon base ^[18]. Human writers use every one of those, and Pew treated them as aggregate evidence rather than a per-page checklist ^[19]. A separate study by researchers at Imperial College London, the Internet Archive and Stanford University reached a similar result using Internet Archive data and several detection methods ^[20].

Watch three things: whether dataset builders start shipping detector scores as per-document metadata instead of silently dropping pages, whether HTML date coverage improves beyond the 10% to 15% the whole finding rests on ^[5], and whether the .org and .edu rates begin tracking .com in later crawls.