Hundreds of OpenAI contractors read whole ChatGPT conversations to score the model's replies
404 Media obtained instruction guides, Slack channels and real prompts from OpenAI's human review program. Reviewers see full chats and a summary of what the user asked for before, with the username stripped.
The Watch · Security desk
What happened
- 404 Media reported that OpenAI is hiring hundreds of contractors to read real ChatGPT prompts, sometimes whole conversations, drawn from a user base the outlet puts at more than 900 million.
- OpenAI said conversations pass through a version of its Privacy Filter model, built to strip personal information, before contractors see them.
- The material 404 Media saw includes instruction guides, Slack channels, the reviewer rating system and real user prompts, which the outlet declined to quote to protect sources.
Why it matters
- Who's exposed exposure Staff who paste client records, contract drafts or incident detail into a personal ChatGPT account are handing that text to a third-party review workflow, because training is on by default on consumer plans.
- What it limits constraint An organisation cannot present the pre-review redaction as a technical safeguard to a client or a regulator when the vendor's own documentation lists missed identifiers and under-redaction as expected failure modes.
- Who has to decide decision Acceptable-use policy has to treat consumer chat as disclosure to a human reviewer, not as a scraping-and-training abstraction, and this stream is separate from the safety review OpenAI has already announced.
- What it sets up precedent With Anthropic confirming human review as well, buyers should assume reviewer access is a category norm and ask for it in writing at contract stage.
Next story · Product · Oct 9
Staff at three Big Five publishers say LLMs now write agent emails and back-cover copyConfirmed2 publishers
A reviewer picks a task from a dashboard and is shown the real prompt of a real ChatGPT user [12]. Sometimes a section above it carries a "user memories summary" describing what that person has previously used the chatbot for, and in some cases where in the world they may live [3]. The username is not there [4]. Anonymization stops at the account label, and the text of the conversation arrives intact [13].
The work runs in three stages: read the prompt, summarize what the reviewer believes the user is asking for, then rate and critique a set of generated responses [14]. One instruction guide 404 Media saw sets the target: "An excellent response should understand the user's intent, provide helpful and accurate assistance, and write in a style that is clear, natural and appropriately warm" [15].
OpenAI's control on the way in is another model. The company told 404 Media that conversations pass through a version of its Privacy Filter, which is designed to detect and remove personal information, before they reach contractors [5]. OpenAI's own page on that model says: "Like all models, Privacy Filter can make mistakes. It can miss uncommon identifiers or ambiguous private references, and it can over- or under-redact entities when context is limited, especially in short sequences" [6]. The vendor describes removal of personal data before human review as best effort [16].
ChatGPT has more than 900 million users, and 404 Media reported that most of them probably do not realize a person may read their conversations [2]. Some of the prompts reviewers see ask ChatGPT to keep the content to itself [7]. "No," a person who works with the prompts said when asked whether users know humans are reading their chats. That person said they did not think users would imagine some contractor somewhere was analyzing the conversations [8].
The reviewers exist because the model has behavioural problems that scraping cannot fix. Internal documents show contractors training ChatGPT not to anthropomorphize itself and to be less sycophantic, a trait that in the 4o model contributed in part to multiple suicides, according to lawsuits cited by 404 Media [17]. This review stream is separate from the chat review OpenAI has announced publicly for users it detects planning to hurt other people [9].
For anyone writing an acceptable-use policy, the relevant line in 404 Media's reporting is that model training is on by default on consumer plans [1]. That account is about consumer tiers; it does not set out how business or enterprise subscriptions are handled. Anthropic confirmed to 404 Media that it also uses human review to improve its models [10], so the same question arises at a different vendor.
Contractors are instructed to escalate tasks they encounter "with potential safety concerns" or personal information [18], and a contractor has read the task before any escalation happens.
What to watch
- Whether OpenAI publishes a user-facing statement that humans may read prompts to improve the model; 404 Media said it put that question to the company.
- Any documentation of whether business and enterprise subscriptions feed the same contractor review pool as consumer plans.
- A data protection complaint that cites the Privacy Filter's own documented under-redaction as the reason de-identification failed.
Clarity's read
What the record supports and how the coverage leans. The claims behind it follow.
Reality
- Evidence78
- Adoption72
- Hype gap+12
- Incentives58
- Confidence74
Claim ledger
Ranked by verification strength, evidence, and original report placement.
- [1]
Model training is turned on by default for consumer ChatGPT plans, per 404 Media's reporting.
- [2]
OpenAI is hiring hundreds of contractors who read a stream of real ChatGPT prompts, which can include whole conversations; ChatGPT has more than 900 million users, most of whom 404 Media says probably do not realize the chats may be read by people.
- [3]
A section above the prompt sometimes includes a "user memories summary" giving an overview of what that user has previously tried to use the chatbot for, and in some cases where in the world the person may live and other personal context.
- [4]
The prompts are anonymized in that the reviewer dashboard does not include the username of the ChatGPT user who entered them, but some prompts can still contain sensitive or personal information.
- [5]
OpenAI told 404 Media that it processes users' conversations through a version of its Privacy Filter model, designed to detect and remove personal information, before they reach the contractors.
- [6]
"Like all models, Privacy Filter can make mistakes. It can miss uncommon identifiers or ambiguous private references, and it can over- or under-redact entities when context is limited, especially in short sequences," a page describing the model on OpenAI's website reads.
- [7]
Some of the prompts indicate the ChatGPT user does not expect a human may read their conversation, because they ask ChatGPT to keep the content to themselves.
- [8]
"No," someone who works with the prompts said when asked if they think ChatGPT users know that humans are reading their chats. "I don't think they would imagine some contractor somewhere [...] is analyzing the conversations."
- [9]
This reading of users' prompts is distinct from publicly announced ChatGPT safety measures, including reviewing chats when the company detects users planning to hurt other people.
- [10]
Anthropic confirmed to 404 Media that it is also using human review to improve its models.
- [11]
404 Media asked OpenAI whether it had explicitly told users that humans may review their prompts in order to improve the model.
- [12]
In a dashboard available to the workers, reviewers select which task to take on and are then presented with the real ChatGPT user's prompt.
- [13]
The only identifier the reviewer dashboard is described as removing is the account username; the conversation content, including the memories summary and any location context, is presented to the reviewer.
- [14]
Contractors work in three stages: reading the real user's prompt; summarizing what they believe the user is asking ChatGPT to do; and rating and critiquing a set of ChatGPT-generated responses.
- [15]
"An excellent response should understand the user's intent, provide helpful and accurate assistance, and write in a style that is clear, natural and appropriately warm," one of the instruction guides reads.
- [16]
Because OpenAI's stated de-identification step is a model its own documentation says can miss identifiers and under-redact, removal of personal information before human review is best effort by the vendor's own account.
- [17]
Internal documents seen by 404 Media show contractors training ChatGPT not to anthropomorphize itself and to be less sycophantic, a problem for OpenAI whose over-sycophantic 4o model led in part to multiple peoples' suicides, according to various lawsuits.
- [18]
An instruction guide for contractors seen by 404 Media tells reviewers to escalate tasks they come across "with potential safety concerns" or personal information.
- [19]
404 Media has seen extensive material related to OpenAI's use of human reviewers, including instruction guides, Slack channels, real ChatGPT user prompts and the rating system reviewers use, and is not quoting the real prompts for source protection reasons.
Sources
2 independent publishers whose own reporting we read for this story.
- 404media.coInside ‘Project Lily’: The Humans Reading Your ChatGPT Chats
2 articles · September 14, 2026
- malwarebytes.comHow to opt out of AI chatbot training
1 article · September 15, 2026
Topics and entities
Topics
- Privacy defaults and opt-outs
- Outsourced data work
- Human Feedback Training
- AI Data Privacy