Overview
A Florida woman, Carli Michelle Heller, is facing a second-degree felony charge after Anthropic’s Claude flagged a diary-style entry in which she allegedly wrote that she planned to attack a Sheriff’s office. Claude’s safety systems escalated the entry to a human reviewer, who judged it a credible threat and reported it to law enforcement. Heller was subsequently detained at her home and charged under Florida Statute 836.10, which criminalises written or electronic threats of violence, mass shootings, or terrorism.
The case is significant not because it represents a technical vulnerability in Claude, but because it exposes a dangerous expectation gap between how users perceive LLM chatbots and how those platforms actually operate.
Technical Analysis
Anthropic’s platform employs automated safety classifiers that scan conversations for content matching threat indicators. When triggered, these systems escalate flagged conversations to human reviewers — a standard human-in-the-loop safety architecture. Anthropic’s terms of service and privacy policy reserve the right to disclose user information in emergencies where disclosure is necessary to prevent death or serious physical injury.
The critical security-relevant issue here is LLM06: Sensitive Information Disclosure. The user’s input — treated by her as private diary content — was retained, processed, reviewed by humans, and disclosed to third parties (law enforcement). This is a direct channel from user input to external disclosure, enabled by the platform’s monitoring infrastructure.
A secondary concern is LLM09: Overreliance. The user overestimated the privacy and confidentiality of the Claude interface, treating it as functionally equivalent to a private journal — a dangerous misperception that the platform’s conversational UX actively encourages.
Framework Mapping
- AML.T0057 – LLM Data Leakage: User-entered data was extracted from the LLM conversation context and disclosed externally, consistent with this technique’s definition even in a sanctioned operational context.
- AML.T0047 – AI-Enabled Product or Service: Claude’s safety pipeline functioned as an AI-enabled monitoring service, triggering real-world consequences from user inputs.
- LLM06 – Sensitive Information Disclosure: Sensitive personal content shared with the LLM was disclosed to law enforcement without the user’s consent.
- LLM09 – Overreliance: The user’s assumption of privacy in an LLM context led directly to her legal exposure.
Impact Assessment
The immediate impact falls on users who share emotionally sensitive, threatening, or legally ambiguous content with LLM platforms under the assumption of privacy. More broadly, this case signals that AI platforms are now active participants in law enforcement referral chains — a function users are largely unaware of.
For enterprises deploying LLMs in HR, mental health support, or employee assistance contexts, this precedent creates significant liability and trust risks if users are not explicitly informed of monitoring and disclosure policies.
Mitigation & Recommendations
- Users: Treat all LLM conversations as potentially reviewable by humans and subject to legal disclosure. Do not use commercial LLM platforms as private journals or for processing distressing thoughts.
- Organisations deploying LLMs: Publish and prominently surface data-sharing and human-review policies at the point of user interaction, not buried in terms of service.
- Platform operators: Consider tiered disclosure frameworks that distinguish between imminent credible threats and ambiguous venting, to avoid over-reporting and erosion of user trust.
- Security teams: Include LLM conversation data retention and disclosure risk in your organisation’s data classification and privacy impact assessments.