Llm-Security

23 reports

All LLM Security Agentic AI Industry News Research Supply Chain Prompt Injection First Look: Security Regulatory Jailbreaks Adversarial ML Data Poisoning Model Theft AI Security Tools Security Operations

Prompt Injection Attacks Manipulate AI Crypto Agents

ATLAS OWASP HIGH ▲ 8.5 SecurityWeek Jul 07, 2026

Researchers identified two active campaigns embedding indirect prompt injection payloads in malicious websites to manipulate autonomous AI agents into executing unauthorised cryptocurrency transactions. The attacks exploit the growing deployment of agentic AI systems that browse the web and take real-world actions with minimal human oversight. This represents a concrete, financially motivated escalation of prompt injection from data exfiltration to direct fund theft.

Claude Opus 4.6 Resists 6,000 Prompt Injection Attempts

ATLAS OWASP MEDIUM ▲ 6.5 Simon Willison Jun 27, 2026

A public challenge exposing an AI email assistant to over 6,000 prompt injection attempts found that Claude Opus 4.6 successfully resisted all efforts to leak secrets or execute malicious instructions embedded in emails. While the result suggests frontier model training against injection attacks is meaningfully improving, security researchers caution that the absence of a successful attack under constrained conditions does not constitute a security guarantee. The author and Hacker News community both note that sophisticated or novel attack vectors could still break through, and irreversible-damage scenarios should not rely solely on model-level defences.

Prompt Injection Malware Evades LLM Security Scanners

ATLAS OWASP HIGH ▲ 8.2 Schneier on Security Jun 25, 2026

A malware developer has embedded nuclear and biological weapons-related text inside JavaScript comment blocks within spyware payloads, specifically to trigger refusal behaviour or context confusion in LLM-powered security analysis pipelines. The technique exploits the architectural gap between how interpreters (which skip comments) and language models (which ingest the full file as input) process the same file. While ineffective against traditional static analysis tooling, the tactic represents a practical adversarial countermeasure targeting AI-first triage workflows and analyst copilots.

Google DeepMind Releases AI Agent Attack Taxonomy

FIRST LOOK ATLAS OWASP HIGH ▲ 8.7 SecurityWeek Jun 25, 2026

Google DeepMind researchers have released a structured taxonomy categorising adversarial attacks against autonomous AI agents into six classes — content injection, semantic manipulation, cognitive state poisoning, behavioural control, systemic, and human-in-the-loop traps — formalising an emerging threat model for agentic AI systems. For defenders, this framework codifies attack paths that exploit the agent's inability to distinguish trusted instructions from attacker-controlled data ingested from web pages, emails, documents, and tool outputs. NIST evaluation data cited in the research shows malicious instruction injection succeeded in 57% of tested agent hijacking scenarios on average, underscoring that these are active, high-yield attack vectors rather than theoretical concerns.

Dragos Launches EmberAI, an OT-Specific AI Platform

FIRST LOOK ATLAS OWASP HIGH ▲ 7.2 SecurityWeek Jun 24, 2026

Dragos has launched EmberAI, an AI module embedded within its OT security platform that allows analysts to query threat intelligence, asset data, and network activity in plain language, grounded in a decade of proprietary OT-specific data. The system introduces new attack surface considerations because it aggregates highly sensitive OT network telemetry, vulnerability data, and adversary intelligence into a single AI-queryable layer — making the platform itself a high-value target. Defenders must weigh the risks of prompt injection, over-reliance on AI-generated recommendations in safety-critical environments, and the intelligence value this consolidated dataset represents to nation-state adversaries.

LLM Role Confusion Attack Bypasses Safety at 61%

ATLAS OWASP HIGH ▲ 8.2 Simon Willison Jun 23, 2026

New research from Ye, Cui, and Hadfield-Menell demonstrates that LLMs prioritise the stylistic format of text over its structural role tags, enabling attackers to craft injected content that mimics internal reasoning blocks and bypasses safety guardrails. The study found attack success rates of 61% when injected text stylistically matched model-internal formats, dropping to just 10% after 'destyling'. The authors conclude that without genuine role perception in models, prompt injection defences will remain fundamentally reactive.

GitHub Ships Data Analytics Agent Built on Copilot

FIRST LOOK ATLAS OWASP MEDIUM ▲ 6.8 GitHub Blog Jun 20, 2026

GitHub has published a detailed engineering account of how it built an internal data analytics agent using GitHub Copilot, exposing the architectural patterns — including natural language-to-SQL translation, autonomous tool invocation, and internal data access — that underpin such systems. For defenders, this blueprint highlights concrete risks around prompt injection into analytics pipelines, excessive agency over sensitive internal datasets, and the challenge of auditing LLM-generated queries before execution. Organisations adopting similar agentic analytics patterns should treat this as a reference threat model rather than a safe-to-copy architecture.

OpenClaw Agent Vulnerable to Prompt Injection RCE

ATLAS OWASP HIGH ▲ 8.5 The Hacker News Jun 12, 2026

Two independent research teams demonstrated that OpenClaw, a self-hosted AI agent, is vulnerable to prompt injection attacks delivered through shared contacts, vCards, location pins, and plain emails — enabling attacker-controlled code execution and sensitive data exfiltration. Imperva's finding, now patched in version 2026.4.23, exploited the agent's failure to mark message objects as untrusted before passing them to the underlying LLM. Varonis separately showed that a single crafted email could instruct an agent to forward mock AWS credentials and customer data to an external address, a behaviour-level risk no patch can fully remediate.

Adversa AI: 89% of AI Agents Fail Security Tests

ATLAS OWASP HIGH ▲ 8.2 SecurityWeek Jun 04, 2026

Adversa AI's AI Risk Quadrant report evaluated 100 AI agents across ten categories, finding that only 11 qualify as both capable and well-defended. The research identifies a structural 'power-protection inversion' where the most capable agents also present the widest attack surface, driven by a 'lethal trifecta' of private data access, exposure to untrusted content, and outbound action capability. Computer and coding agents showed the most severe exposure, raising urgent concerns about autonomous agent deployment in enterprise environments.

SentinelOne Warns on Prompt Injection Risks in AI Agents

ATLAS OWASP MEDIUM ▲ 6.5 SentinelOne Blog May 25, 2026

SentinelOne has published guidance on securing agentic AI systems, framing unverified trust in AI agents as a core enterprise risk. The piece promotes their Prompt Security product as a control layer for AI tools, agents, and pipelines deployed across the enterprise. While primarily a product-focused announcement, it highlights the genuine security challenge of blind trust in autonomous AI agents executing actions on behalf of users and systems.

Agent Hijacking: Microsoft's Defense-in-Depth Framework

ATLAS OWASP MEDIUM ▲ 7.2 Microsoft Security Blog May 15, 2026

Microsoft's Security Blog introduces a layered defense-in-depth model specifically designed for autonomous AI agents, which now invoke tools, modify data, and trigger workflows with minimal human oversight. The framework identifies novel threat classes — including agent hijacking, intent breaking, and supply chain compromise — that are amplified by agentic autonomy. The guidance positions application-layer architecture, permissions, and governance as the most critical controls as agent autonomy scales.

Excessive Agency in AI Agents: Tool Access Control Gaps

ATLAS OWASP LOW ▲ 6.2 HN AI Security May 13, 2026

Statewright is an open-source framework that enforces state machine constraints on AI agents, restricting which tools agents can invoke during each phase of a workflow. The project directly addresses the Excessive Agency problem, where AI agents operating with broad, unconstrained tool access can take unintended or harmful actions. While a defensive development rather than a threat disclosure, it signals growing practitioner awareness of agentic AI risk and offers a concrete mitigation pattern for teams deploying coding agents like Claude Code, Codex, or Cursor.

CrowdStrike Red Teaming: LLM Jailbreak and Data Poisoning

ATLAS OWASP MEDIUM ▲ 6.5 SecurityWeek May 06, 2026

Joey Melo, Principal Security Researcher at CrowdStrike, outlines his methodology for AI red teaming, focusing on manipulating LLM guardrails through jailbreaking and data poisoning without altering underlying source code. His work, rooted in competitive AI hacking challenges, translates classical adversarial thinking into the emerging field of machine learning security. The profile highlights the growing professionalisation of AI red teaming as organisations seek to harden LLM deployments against real-world manipulation attacks.

AI Agents Exploit Excessive Agency to Delete Production Databases

ATLAS OWASP HIGH ▲ 6.5 Dark Reading May 02, 2026

Organisations are deploying AI agents into production environments without adequate security testing, resulting in destructive outcomes such as unintended deletion of production databases. The core risk is excessive agency granted to AI systems before trust boundaries and guardrails are established. This represents a systemic industry failure to apply basic security principles before integrating autonomous AI tooling into critical infrastructure.

Amazon Bedrock Prompt Injection Traverses Agent Hierarchies

ATLAS OWASP HIGH ▲ 8.5 Palo Alto Unit 42 Apr 23, 2026

Unit 42 researchers conducted red-team analysis of Amazon Bedrock's multi-agent collaboration framework, demonstrating how attackers can systematically exploit prompt injection to traverse agent hierarchies, extract system instructions, and invoke tools with attacker-controlled inputs. The research reveals that multi-agent architectures introduce compounded attack surfaces through inter-agent communication channels, though no underlying Bedrock vulnerabilities were identified. Properly configured Guardrails and pre-processing stages effectively mitigate the demonstrated attack chains.

Llm-Security

Stay ahead of the threat.