Overview
Threat actors are abusing legitimate sharing and publishing features built into major AI platforms — including Anthropic’s Claude, OpenAI’s ChatGPT, and xAI’s Grok — to distribute malware to unsuspecting users. Research published by the Huntress Security Operations Centre (SOC) on 11 September 2026 documents a nine-month tracking effort revealing that attackers are not targeting AI model internals or training pipelines; instead, they are exploiting the trust users place in well-known AI domains to make malicious content appear legitimate.
The core insight is straightforward but significant: when a download or instruction appears on claude.ai, chatgpt.com, or grok.com, users are far less likely to apply standard scepticism. Attackers are deliberately exploiting that cognitive shortcut.
Technical Analysis
Huntress identified three primary abuse vectors:
Claude Artifacts — Claude can generate interactive content displayed in a preview pane and published via a public link. Anthropic applies only generic disclaimers rather than deep content vetting, leaving a window for malicious actors to embed harmful instructions or download prompts inside polished, plausible-looking Artifacts.
Shareable claude.ai/share and chatgpt.com/grok.com conversation URLs — Published conversations are indexable by search engines. Attackers post malicious shared conversations to crawlable forums or social media, causing them to surface in results for common troubleshooting queries. Victims following a search result land on an official-looking AI domain hosting attacker-controlled content.
FakeAgent Campaign (July 2026) — The most concrete example documented, FakeAgent struck more than 29 organisations. It originated with a malicious Claude Artifact hosted directly on claude.ai, using the platform’s own branding and UI framing to lend the malicious content legitimacy. The campaign was short-lived by design — these operations often run for only hours or days before providers remove the content — but the window was sufficient to compromise multiple targets.
The attack lifecycle follows a pattern: publish → index or share → victim clicks a recognisable AI platform link → malware delivered under trusted branding → provider eventually removes content.
Framework Mapping
- AML.T0067 (LLM Trusted Output Components Manipulation) — Directly applicable; attackers manipulate what Claude renders in Artifacts to serve malicious ends.
- AML.T0077 (LLM Response Rendering) — The browser rendering of AI-generated content is the delivery mechanism.
- AML.T0114 (AI Service Web Interface) — The AI platform’s own web interface is the attack surface.
- AML.T0115 (Publish Poisoned AI Artifacts) — Malicious Artifacts are published to Anthropic’s infrastructure.
- LLM02 (Insecure Output Handling) — AI-generated content is rendered without sufficient sanitisation or vetting.
- LLM09 (Overreliance) — Users over-trust AI platform branding, reducing critical scrutiny of content sourced from those domains.
Impact Assessment
At least 29 organisations were confirmed affected by the FakeAgent campaign alone. The broader population at risk includes any user or organisation that follows shared AI content links — particularly those using AI platforms for technical troubleshooting, where malicious content can masquerade as helpful guides or tool downloads. The short campaign windows make retroactive detection difficult and mean traditional reputation-based defences are often ineffective.
Mitigation & Recommendations
- Perimeter controls: Evaluate whether broad access to AI platform share URLs (e.g.,
claude.ai/share/*,chatgpt.com/share/*) is necessary; restrict where operationally feasible. - User awareness: Train staff to treat links from AI platforms with the same scrutiny as any third-party URL — platform branding is not content endorsement.
- Endpoint detection: Ensure EDR solutions cover malware delivery through browser-rendered content, including short-lived pages.
- Threat intelligence monitoring: Subscribe to feeds tracking abuse of AI platform sharing features; Huntress and similar SOC providers are actively tracking these campaigns.
- Zero-trust content review: Apply download scanning even when the referring domain is a well-known AI service.