Research

121 reports

All LLM Security Agentic AI Industry News Research Supply Chain Prompt Injection First Look: Security Regulatory Jailbreaks Adversarial ML Data Poisoning Model Theft AI Security Tools Security Operations

Claude Code Indirect Prompt Injection Spawns Reverse Shell

ATLAS OWASP HIGH ▲ 8.5 SecurityWeek Jun 30, 2026

Researchers have demonstrated that indirect prompt injection attacks embedded within seemingly benign code repositories can cause Claude Code — Anthropic's agentic coding assistant — to spawn a reverse shell on a developer's machine. The attack exploits Claude Code's autonomous execution capabilities, using hidden instructions in repository content to hijack the host system without any explicit user consent. This highlights a critical risk in agentic AI tools that operate with elevated system privileges in developer environments.

NanoEuler Launches GPT-2 LLM Built from Scratch in C/CUDA

FIRST LOOK ATLAS OWASP MEDIUM ▲ 5.8 Cohere AI (via HN) Jun 29, 2026

NanoEuler is an open-source GPT-2-class language model (~116M parameters) built entirely from scratch in C/CUDA, including hand-written backpropagation, a BPE tokenizer, FlashAttention, pretraining, and supervised fine-tuning — with RLHF/DPO planned. For defenders, the significance lies in the democratisation of low-level, dependency-free LLM training infrastructure: adversaries gain a highly portable, auditable, and modifiable training stack that bypasses standard ML framework telemetry and supply chain controls. Security teams should treat this class of 'from-scratch' open-source LLM tooling as a potential foundation for covert fine-tuning pipelines, backdoor insertion, and evasion of model-level safety controls.

Zhipu AI Releases GLM-5.2 Open-Weight Model

FIRST LOOK ATLAS OWASP HIGH ▲ 8.2 The Verge AI Jun 29, 2026

Zhipu AI (Z.ai) has released GLM-5.2, an open-weight model that researchers report matches Anthropic's Mythos in bug-finding and cybersecurity-related tasks, while remaining freely downloadable and runnable on commodity hardware. The open-weight distribution removes access controls and usage monitoring that restrict frontier closed models, enabling unconstrained offensive security use by any actor. Defenders face a materially elevated threat from nation-state and cybercriminal actors who can now fine-tune, deploy, and weaponise a frontier-class vulnerability-discovery model without API gatekeeping or usage telemetry.

Claude Code Prompt Injection via GitHub Supply Chain

ATLAS OWASP HIGH ▲ 9.1 BleepingComputer Jun 29, 2026

Mozilla 0DIN researchers demonstrated a novel attack chain in which a seemingly clean GitHub repository tricks AI coding agents like Claude Code into executing a reverse shell payload — with no malicious code ever present in the repo itself. The attack leverages three innocuous components: a Python package that deliberately errors on first run, an error message that instructs the agent to run an init command, and a shell script that fetches and executes a payload stored in an attacker-controlled DNS TXT record. The technique exploits the autonomous error-recovery behaviour of agentic AI tools, effectively turning a safety feature into an attack vector.

OpenAI Releases GPT-5.6 Sol for Vulnerability Research

FIRST LOOK ATLAS OWASP HIGH ▲ 8.2 The Hacker News Jun 29, 2026

OpenAI has released a limited preview of GPT-5.6 Sol, Terra, and Luna to select partners, positioning Sol as its most capable model for vulnerability research and exploit chain development, benchmarked against real-world hardened targets via an internal framework called VulnLMP. The model's demonstrated ability to produce credible memory safety leads and automate substantial portions of vulnerability research pipelines materially lowers the barrier for both defenders and adversaries. Security teams should expect accelerated attacker timelines for exploit development and increased pressure on detection and patch-deployment cadences.

AI Code Review Agents: DoS Loop Costs $41K in Inference

ATLAS OWASP MEDIUM ▲ 6.5 Simon Willison Jun 27, 2026

A hypothetical but technically grounded incident report depicts two competing AI code review agents entering an uncontrolled disagreement loop over a suspected malicious package, generating 340 comments and $41,255 in inference costs before human intervention. The scenario illustrates real risks of excessive agency, lack of circuit-breakers, and cost-based denial-of-service in multi-agent agentic pipelines. While fictional, the scenario directly mirrors documented failure modes in production AI systems and supply chain security workflows.

Claude Opus 4.6 Resists 6,000 Prompt Injection Attempts

ATLAS OWASP MEDIUM ▲ 6.5 Simon Willison Jun 27, 2026

A public challenge exposing an AI email assistant to over 6,000 prompt injection attempts found that Claude Opus 4.6 successfully resisted all efforts to leak secrets or execute malicious instructions embedded in emails. While the result suggests frontier model training against injection attacks is meaningfully improving, security researchers caution that the absence of a successful attack under constrained conditions does not constitute a security guarantee. The author and Hacker News community both note that sophisticated or novel attack vectors could still break through, and irreversible-damage scenarios should not rely solely on model-level defences.

Prompt Injection Malware Evades LLM Security Scanners

ATLAS OWASP HIGH ▲ 8.2 Schneier on Security Jun 25, 2026

A malware developer has embedded nuclear and biological weapons-related text inside JavaScript comment blocks within spyware payloads, specifically to trigger refusal behaviour or context confusion in LLM-powered security analysis pipelines. The technique exploits the architectural gap between how interpreters (which skip comments) and language models (which ingest the full file as input) process the same file. While ineffective against traditional static analysis tooling, the tactic represents a practical adversarial countermeasure targeting AI-first triage workflows and analyst copilots.

Google DeepMind Releases AI Agent Attack Taxonomy

FIRST LOOK ATLAS OWASP HIGH ▲ 8.7 SecurityWeek Jun 25, 2026

Google DeepMind researchers have released a structured taxonomy categorising adversarial attacks against autonomous AI agents into six classes — content injection, semantic manipulation, cognitive state poisoning, behavioural control, systemic, and human-in-the-loop traps — formalising an emerging threat model for agentic AI systems. For defenders, this framework codifies attack paths that exploit the agent's inability to distinguish trusted instructions from attacker-controlled data ingested from web pages, emails, documents, and tool outputs. NIST evaluation data cited in the research shows malicious instruction injection succeeded in 57% of tested agent hijacking scenarios on average, underscoring that these are active, high-yield attack vectors rather than theoretical concerns.

Anthropic's Mythos AI Breached Classified US Government Systems in Hours

ATLAS OWASP CRITICAL ▲ 9.1 SecurityWeek Jun 24, 2026

Anthropic's Mythos AI model identified vulnerabilities in classified US government computer systems within hours during a government-sanctioned testing exercise under Project Glasswing. A senior US official confirmed the findings to the Associated Press, corroborating statements made by Sen. Mark Warner that the model 'broke into almost all of our classified systems.' The incident marks a landmark demonstration of AI-enabled offensive cyber capability at the highest sensitivity levels of government infrastructure.

LLM Role Confusion Attack Bypasses Safety at 61%

ATLAS OWASP HIGH ▲ 8.2 Simon Willison Jun 23, 2026

New research from Ye, Cui, and Hadfield-Menell demonstrates that LLMs prioritise the stylistic format of text over its structural role tags, enabling attackers to craft injected content that mimics internal reasoning blocks and bypasses safety guardrails. The study found attack success rates of 61% when injected text stylistically matched model-internal formats, dropping to just 10% after 'destyling'. The authors conclude that without genuine role perception in models, prompt injection defences will remain fundamentally reactive.

OpenAI's ChatGPT Image Generation Fails Content Moderation

FIRST LOOK ATLAS OWASP HIGH ▲ 8.2 OpenAI (via HN) Jun 22, 2026

Mindgard researchers demonstrated that ChatGPT's image generation pipeline can be manipulated through an indirect, socially-engineered prompt to produce violent and sexually explicit content without users directly requesting it, exposing a significant failure in OpenAI's content moderation controls. Defenders and enterprise operators of ChatGPT-integrated products face a newly validated attack class where innocuous-looking prompt patterns — potentially spreading virally — can systematically strip safety guardrails from image generation. This finding signals that content filter bypasses in multimodal systems are reproducible at scale, raising urgent questions about the adequacy of output-layer filtering as a sole defence mechanism.

AutoGen Studio RCE: AutoJack Exploit Chain Targets Developers

ATLAS OWASP HIGH ▲ 9.1 The Hacker News Jun 20, 2026

Microsoft researchers disclosed AutoJack, an exploit chain targeting AutoGen Studio's MCP WebSocket endpoint that allows a single malicious web page to execute arbitrary commands on a developer's host machine via an AI browsing agent. The attack chains three distinct weaknesses — localhost trust bypass, missing authentication on MCP paths, and unsanitised command execution — requiring no credentials or user interaction beyond the agent loading the attacker's URL. While the vulnerable handler was not included in stable PyPI releases, it shipped in two pre-release builds that remain unyanked, leaving anyone who installed those versions exposed.

Microsoft AutoGen Studio RCE via MCP Bypass

ATLAS OWASP HIGH ▲ 9.1 Microsoft Security Blog Jun 19, 2026

Researchers at Microsoft identified a three-stage exploit chain in AutoGen Studio that allows a malicious web page visited by a browsing AI agent to reach the host's local Model Context Protocol (MCP) WebSocket and spawn arbitrary processes. The chain exploits a bypassable origin allowlist, authentication middleware that excluded MCP endpoints, and unsanitised URL-derived command parameters. Although the vulnerable surface was never shipped in a PyPI release, the finding exposes a systemic architectural risk in any agent framework that combines untrusted browsing with privileged localhost services.

OpenAI Launches ChatGPT for Science with Institutional Access

FIRST LOOK ATLAS OWASP MEDIUM ▲ 5.8 BleepingComputer Jun 18, 2026

OpenAI is internally testing a specialised 'ChatGPT for Science' subscription tier, likely restricted to verified universities and research institutions, building on capabilities from GPT-Rosalind — a purpose-built life sciences model already deployed under a trusted-access structure with select pharma partners. The gated, domain-specific nature of this offering creates novel identity and access verification attack surfaces, as threat actors will likely probe credential and institutional verification mechanisms to gain privileged access to specialised scientific knowledge. Defenders at academic and research institutions should anticipate increased phishing campaigns targeting institutional credentials and prepare governance frameworks for AI use in sensitive research environments.

Research

Stay ahead of the threat.