LIVE THREATS
HIGH Discord Sleuths Gained Unauthorized Access to Anthropic’s Mythos // HIGH GTIG AI Threat Tracker: Distillation, Experimentation, and (Continued) Integration of AI … // MEDIUM Open source memory layer so any AI agent can do what Claude.ai and ChatGPT do // MEDIUM Python package 'llm-openai-via-codex 0.1a0' hijacks Codex CLI // CRITICAL LMDeploy CVE-2026-33626 Flaw Exploited Within 13 Hours of Disclosure // HIGH Show HN: Browser Harness – Gives LLM freedom to complete any browser task // CRITICAL Paloalto's Zealot successfully attacks misconfigured cloud environments // HIGH Bitwarden CLI Compromised in Ongoing Checkmarx Supply Chain Campaign // HIGH Bad Memories Still Haunt AI Agents // CRITICAL ChatGPT's code runtime silently exfiltrates user data via malicious prompt //
ATLAS OWASP HIGH Significant risk · Prioritise patching Model Theft Agentic AI LLM Security Adversarial ML Research Industry News RELEVANCE ▲ 8.5

GTIG AI Threat Tracker: Distillation, Experimentation, and (Continued) Integration of AI for Adversarial Use

SOURCE Mandiant Blog ↗
TL;DR HIGH
  • What happened: Nation-states and cybercriminals are actively distilling proprietary LLMs and integrating AI into malware and phishing operations.
  • Who's at risk: AI model providers and enterprises are most exposed — model IP is being stolen via distillation attacks while their staff face increasingly convincing AI-generated phishing.
  • Act now: Implement rate-limiting and anomaly detection on model inference APIs to detect distillation attempts · Train security teams to recognise AI-augmented spear-phishing lures with unusually polished language and contextual accuracy · Monitor for unauthorised use of generative AI APIs (e.g., Gemini, OpenAI) within malware callout patterns and network telemetry
GTIG AI Threat Tracker: Distillation, Experimentation, and (Continued) Integration of AI for Adversarial Use

Overview

Google Threat Intelligence Group (GTIG) and Google DeepMind’s Q4 2025 AI Threat Tracker documents a clear escalation in the adversarial use of artificial intelligence. Three primary trends dominate the findings: a surge in model extraction or “distillation” attacks targeting proprietary AI systems, the deeper integration of LLMs into nation-state offensive operations, and the emergence of malware families — notably HONESTCUE — that actively call generative AI APIs to generate malicious code at runtime. While GTIG stops short of declaring a fundamental shift in the threat landscape, the report marks a qualitative step-change in how sophisticated actors operationalise AI.

Technical Analysis

Model Extraction (Distillation Attacks): Distillation attacks involve querying a target model repeatedly via its inference API to generate input-output pairs, which are then used to train a surrogate model that approximates the original’s behaviour. Google DeepMind and GTIG observed a significant increase in such activity from private sector entities and researchers attempting to clone proprietary model logic — a clear violation of terms of service and a form of intellectual property theft. Google states it has detected, disrupted, and mitigated these extraction campaigns.

Nation-State LLM Integration: Actors attributed to DPRK, Iran, PRC, and Russia were observed using LLMs to accelerate reconnaissance, craft highly contextualised phishing lures, and conduct technical research. These use cases represent productivity amplification rather than novel capabilities — LLMs reducing the time and skill required to produce credible social engineering content.

AI-Integrated Malware — HONESTCUE: Perhaps the most technically notable finding is the HONESTCUE malware family, which experiments with calling Gemini’s API to dynamically generate code enabling file download functionality. This marks an early but significant indicator of malware leveraging live LLM inference as part of its execution chain, rather than static AI-generated code embedded at compile time. Agentic AI patterns are beginning to appear in adversarial tooling development workflows.

Framework Mapping

  • AML.T0040 / LLM10 (Model Theft): Distillation attacks directly map to model extraction and theft of proprietary intellectual property via inference API abuse.
  • AML.T0047 / LLM08 (Excessive Agency): HONESTCUE’s use of Gemini’s API at runtime represents an agentic pattern where the malware delegates code generation to an external LLM.
  • AML.T0043: Nation-state actors crafting nuanced phishing lures via LLM constitute adversarial data crafting for social engineering purposes.
  • LLM06 (Sensitive Information Disclosure): Model distillation implicitly risks exposure of training data characteristics embedded in model behaviour.

Impact Assessment

AI model providers face direct IP theft risk from organised distillation campaigns. Enterprises are increasingly exposed to AI-augmented phishing that is harder to detect through traditional indicators. The emergence of malware that calls live LLM APIs introduces a new detection gap — security tooling trained on static signatures will miss dynamic, AI-generated payload components. Nation-state actors from four major adversary blocs have now demonstrably operationalised LLMs, broadening the attack surface significantly.

Mitigation & Recommendations

  • Rate-limit and fingerprint inference API access to detect systematic distillation patterns (high query volume, structured input diversity).
  • Block or alert on outbound LLM API calls from endpoints and servers where such traffic is unexpected — a key indicator of HONESTCUE-style malware.
  • Invest in AI-generated content detection for inbound email and communications, particularly for high-value targets susceptible to spear-phishing.
  • Update threat models to include agentic AI misuse scenarios in red team exercises and tabletop simulations.
  • Monitor for ToS violations and enforce strict API key lifecycle management to limit misuse surface.

References