LIVE FEED
CRITICAL OpenAI Agent Swarm Attacked RubyGems Supply Chain in May // HIGH Meta Faces Lawsuit Over Biometric Data Harvesting for AI Training // CRITICAL Claude Weaponised by State Hackers for Automated Data Theft // LOW Hugging Face security.txt Redirects AI Agents Away From Live Systems // HIGH Houthi Users Weaponised Claude AI for Advanced Arms Dev // CRITICAL Claude Abused by ShinyHunters to Scan 1.8M Android APKs // HIGH Attackers Abuse Claude Artifacts and ChatGPT Links to Spread Malware // FIRST LOOK OpenAI Astra Gains End-to-End Trust for Production Systems // CRITICAL APT29 Abuses Claude to Auto-Rebuild Malware on Detection // HIGH AI Agent Builds Self-Expanding Stolen LLM Inference Supply Chain //
ATLAS OWASP HIGH Significant risk · Prioritise patching Agentic AI Supply Chain LLM Security Model Theft RELEVANCE ▲ 9.2

AI Agent Builds Self-Expanding Stolen LLM Inference Supply Chain

SOURCE SANS Internet Storm Center ↗
TL;DR HIGH
  • What happened: An autonomous agent harvested stolen LLM API keys and aggregated them into a self-expanding attacker-controlled inference gateway.
  • Who's at risk: LLM resale platform operators and API gateway providers with weak authentication or open registration are directly exposed to credential harvesting and inference theft.
  • Act now: Audit all LLM API gateway endpoints for unauthenticated model catalog exposure and default credential risks · Implement rate limiting, anomaly detection, and billing alerts on inference endpoints to detect harvesting behaviour · Deploy honeypot LLM endpoints to surface attacker playbooks and enumerate active offensive infrastructure
AI Agent Builds Self-Expanding Stolen LLM Inference Supply Chain

Overview

A SANS Internet Storm Center researcher operating an AI honeypot captured a sophisticated, semi-autonomous coding agent conducting a complete offensive inference supply chain operation. Unlike conventional credential theft, the attacker built a feedback loop: stolen LLM API access was consolidated into an attacker-controlled gateway, which in turn powered further reconnaissance and harvesting. The operation represents a maturing class of AI-native cybercrime in which the commodity being stolen is inference capacity itself.

Technical Analysis

The honeypot emulated an OpenAI-compatible inference endpoint. When the agent called it as a free LLM backend, it inadvertently transmitted approximately 43 KB of its own control-plane context — including an AGENTS.md file, an offensive playbook, infrastructure notes, reconnaissance scripts, harvested API keys, and prior target lists.

The captured workflow followed four stages:

1. Discovery. The agent generated FOFA search queries (title="V2Board", header="subscription-userinfo") to locate LLM resale gateways and subscription infrastructure exposed on the open internet.

2. Access Acquisition. The playbook enumerated multiple acquisition vectors: open registration with free credit balances, default credentials, group_id authorization weaknesses, exposed endpoints such as /api/auth-files, and automated trial-account farming using temporary email services and CAPTCHA-solving APIs.

3. Inference Validation. Harvested keys were tested against resale endpoints advertising current premium model names (e.g., claude-opus-5). The researcher notes that model name labels are reseller claims, not verified model identities. Validation evolved over sessions to include a functional code-logic test — asking each endpoint to compute a factorial — to distinguish live inference from canned responses.

4. Aggregation and Re-Serving. A second capture session documented the attacker standing up a self-hosted New-API instance to aggregate validated keys behind a single unified gateway, completing the supply chain loop.

A critical operational security failure was also captured: the playbook instructed the agent to verify its proxy was active before attacking by comparing the proxied IP against a reference value — which was the operator’s real, unproxied egress IP, exposing it directly to the honeypot.

Framework Mapping

This operation maps to several MITRE ATLAS techniques: AML.T0040 (AI Model Inference API Access) and AML.T0010 (AI Supply Chain Compromise) cover the core theft and aggregation. AML.T0103 (Deploy AI Agent) and AML.T0098 (AI Agent Tool Credential Harvesting) describe the autonomous execution layer. AML.T0057 (LLM Data Leakage) captures the inadvertent control-plane exposure via the honeypot. On the OWASP side, LLM05 (Supply Chain Vulnerabilities), LLM08 (Excessive Agency), and LLM10 (Model Theft) are the most directly applicable categories.

Impact Assessment

LLM resale platform operators face direct financial harm through API quota exhaustion and fraudulent account farming. Downstream consumers of aggregated inference may unknowingly route traffic through attacker-controlled infrastructure. The self-expanding nature of the supply chain means scale can grow without proportional attacker effort, and the pattern is likely to be replicated as inference costs remain a meaningful operational expense.

Mitigation & Recommendations

  • Harden gateway authentication: Disable open registration with free balances; enforce identity verification and spending caps.
  • Protect internal endpoints: Restrict /api/auth-files and similar administrative endpoints behind authenticated access controls.
  • Monitor for harvesting signals: Implement anomaly detection on key validation patterns, rapid factorial-style probe requests, and bulk model catalog enumeration.
  • Deploy inference honeypots: Emulated OpenAI-compatible endpoints can surface attacker playbooks, tooling, and operational infrastructure at low cost.
  • Audit agent context boundaries: Ensure autonomous coding agents do not embed sensitive credentials or playbook data in model request context.

References

◉ AI THREAT BRIEFING

Stay ahead of the threat.

Twice-weekly digest of critical AI security developments — every story mapped to MITRE ATLAS and OWASP LLM Top 10. Free.

No spam. Unsubscribe anytime.