Project Lily: Human Review of ChatGPT Conversations and the Privacy Risks Involved

On September 14, 2026, investigative reporting by 404 Media revealed that OpenAI runs a large‑scale human‑in‑the‑loop operation called Project Lily. According to internal documents obtained by the outlet, the programme hires hundreds of contract workers who are given access to authentic user queries and the full conversation history generated by ChatGPT. Their task is to summarise the user’s intent, evaluate the relevance of several model responses and flag language that sounds overly artificial, anthropomorphic or excessively agreeable.

The reviewers work from a stripped‑down view of the chat. OpenAI claims that usernames are removed and that every piece of text passes through a so‑called Privacy Filter designed to redact personally identifiable information. The filter is advertised as a data‑minimisation tool, not a blanket guarantee of anonymity. OpenAI acknowledges that the filter can miss sensitive details, especially when users disclose professional contexts, general locations or personal life events within their prompts.

What the announcement says

Project Lily’s guidelines explicitly aim to improve the model’s tone and reduce the illusion that it possesses human‑like agency. Reviewers are instructed to note instances where the model sounds too confident, offers unsolicited advice, or repeats the same phrasing across different sessions. The feedback loop feeds into OpenAI’s continuous‑learning pipeline, shaping future model updates.

The scope of the programme is not universal. According to Tom’s Guide, the "Improve the model for everyone" setting is enabled by default for personal accounts—Free, Plus and Pro—while it is turned off for Enterprise, Business and Education tiers. Importantly, the setting only influences conversations that occur after it is enabled; past chats are not retroactively re‑processed under the new consent framework.

The limits that still matter

Understanding the privacy implications requires a clear distinction between several technical concepts:

• Pseudonymisation – replacing direct identifiers with pseudonyms while retaining the possibility of re‑identification through additional data.
• Anonymisation – removing or altering data so that individuals cannot be identified, even with auxiliary information.
• Consent – the explicit permission a user gives for their data to be used in a particular way.
• Human control – the involvement of people in reviewing, correcting or making decisions about data handling.

Project Lily operates somewhere between pseudonymisation and full anonymisation. The removal of usernames and the application of the Privacy Filter constitute a form of pseudonymisation, but the retained conversational content can still expose indirect identifiers. Because the contractors are not the original users, consent is effectively inferred from the platform’s terms of service rather than obtained on a per‑conversation basis.

What organizations should verify

The presence of a human review layer also raises concerns about data security and insider risk. Contractors handle large volumes of user‑generated text, some of which may contain confidential business information or personal health details. While OpenAI states that contractors are bound by confidentiality agreements, the sheer scale of the operation makes comprehensive oversight challenging.

Users can mitigate exposure by adjusting their settings. The "Improve the model for everyone" toggle can be disabled in the account preferences for personal plans. For higher‑tier accounts, the default off‑state already limits the flow of data to human reviewers. Additionally, users should avoid sharing sensitive personal or professional details in prompts, especially when using free or Plus accounts where the review programme is active.

In summary, Project Lily illustrates the trade‑off between model quality and privacy. OpenAI’s automated filters provide a first line of protection, but they are not infallible, and human reviewers still see enough context to potentially infer personal information. The programme’s selective activation means that not every user is subject to the same level of scrutiny, underscoring the importance of informed consent and transparent settings for anyone relying on ChatGPT for confidential tasks.