Overview
Multiple users on Hacker News are reporting that OpenAI’s ‘allow training’ setting — which allows user conversation data to be used for model training — is being silently re-enabled after users have deliberately turned it off. The original poster, jacquesm, noted they had reset the setting more than once and kept a careful record of when they last disabled it, only to find it re-enabled on a subsequent check. A corroborating report indicates the same behaviour has been observed on Anthropic’s Claude platform, specifically following a subscription renewal.
This behaviour, whether intentional or a result of software bugs triggered by TOS updates or subscription events, undermines the foundational principle of informed consent in AI data governance.
Technical Analysis
The mechanism behind the reset is not publicly confirmed, but community analysis points to several plausible triggers:
- TOS or subscription renewal events: Accepting a new terms-of-service agreement or resubscribing to a service tier may silently reset privacy preferences to their defaults (opt-in).
- Account-level configuration resets: Platform updates may not preserve user-level privacy flag states, reverting them to the system default.
- Security classifier override: One commenter noted that even with opt-out enabled, internal security classifiers may flag conversations and route them to internal model improvement pipelines regardless of user preference.
The net effect is that an ‘opt-out’ functions more as a temporary pause than a durable preference, meaning user data may be incorporated into training datasets during windows where the setting has silently reverted.
Framework Mapping
- AML.T0020 – Poison Training Data / AML.T0059 – Erode Dataset Integrity: If user data re-enters training pipelines without consent, it may include sensitive, proprietary, or adversarially crafted content that degrades or manipulates model integrity.
- AML.T0018 – Manipulate AI Model: Persistent unauthorised data collection can shift model behaviour in ways not anticipated or audited.
- LLM03 – Training Data Poisoning: The unconsented ingestion of user conversations represents a training data governance failure, creating exposure to data quality and integrity risks.
- LLM06 – Sensitive Information Disclosure: Users who believed their data was excluded may have shared confidential information under a false assumption of privacy.
Impact Assessment
The impact is broad. Any OpenAI or Claude subscriber who opted out of training data sharing and handles sensitive conversations — including legal, medical, financial, or proprietary business data — may have unknowingly contributed that data to training corpora. There is no confirmed mechanism for retroactive removal of data already ingested. The community observation that ‘opt-out is more like a pause button’ suggests this is a systemic design issue rather than an isolated bug.
Mitigation & Recommendations
- Audit settings immediately: Check your OpenAI Data Controls and Claude Privacy settings now. Re-disable training data sharing and screenshot the state with a timestamp.
- Monitor after any TOS update or subscription event: Treat account-level changes as triggers to re-verify privacy preferences.
- Assume opt-outs are not durable: Operationally treat any LLM platform as potentially ingesting your data unless independently verified.
- Request third-party audit: Advocate for independent auditing of training data opt-out mechanisms as called for by community members.
- Limit sensitive data exposure: Do not share confidential, proprietary, or personally identifiable information in LLM sessions without a verified, persistent privacy guarantee.