Overview
ChatPRD, an AI-powered product management platform built on top of large language model APIs, has been subjected to a sustained wave of token fraud. Operators created looping accounts using free trials and promotional codes to tunnel into the platform’s underlying LLM agent, consuming inference tokens without payment. The abuse ranged from opportunistic free AI usage to organised commercial operations — including at least one group reportedly funding a Roblox and Fortnite map development studio by reselling access to the hijacked agent. The platform’s founder, Claire Vo, disclosed the abuse publicly and announced the discontinuation of free trials.
Technical Analysis
The attack pattern is straightforward but effective. Adversaries automate account creation across free trial tiers, cycling through promo codes to maintain access without triggering billing. The LLM agent is then accessed programmatically — either directly or via tunnelling — to serve downstream users or automated workflows at no cost to the attacker.
A secondary technique observed was model fingerprinting via probe prompts. Attackers submitted distinctive creative prompts — specifically SVG images of pelicans riding bicycles — to characterise the response output of the underlying model. Different model families (GPT-4, Claude, Gemini, etc.) produce subtly different outputs for unusual creative tasks, making this a reliable heuristic for identifying the model powering a wrapped product. This is a documented reconnaissance technique used by researchers to identify stealth preview models on platforms such as OpenRouter.
This fingerprinting behaviour maps to active probing of LLM system information and output characteristics, and may precede further exploitation such as jailbreak attempts tailored to the identified model family.
Framework Mapping
- AML.T0012 – Valid Accounts: Attackers created legitimate-appearing trial accounts to gain authorised access to the platform.
- AML.T0040 – AI Model Inference API Access: The core objective was unauthorised consumption of inference capacity.
- AML.T0063 – Discover AI Model Outputs / AML.T0069 – Discover LLM System Information: Probe prompts were used to fingerprint the underlying model.
- LLM04 – Model Denial of Service: High-volume fraudulent usage degrades service availability and inflates costs for legitimate users.
- LLM10 – Model Theft: Reselling API access constitutes indirect monetisation of the operator’s model investment.
Impact Assessment
The direct financial impact on ChatPRD is significant: the operator is bearing inference costs for usage that generates no revenue. The indirect impact — forced removal of free trials — eliminates a key customer acquisition channel. For the broader AI SaaS ecosystem, this case demonstrates that any platform offering low-friction LLM access is a viable target for token fraud at scale. Smaller operators without sophisticated abuse detection are particularly vulnerable.
Mitigation & Recommendations
- Enforce verified identity on trial access — require phone or payment verification before granting any inference quota, even on free tiers.
- Implement per-account rate limits and anomaly detection — flag accounts exhibiting looping, scripted, or high-volume prompt behaviour for manual review.
- Monitor for fingerprinting probes — unusual creative prompt patterns (highly distinctive, repeated across accounts) may indicate model enumeration activity.
- Apply token budgets at the session and account level — hard caps prevent runaway consumption even if abuse is not immediately detected.
- Rotate and scope API keys — ensure that compromised or abused trial credentials cannot escalate to production inference endpoints.
References
- Simon Willison’s Weblog: https://simonwillison.net/2026/Nov/20/token-fraud-is-here-and-its-bad