Prompt-Injection

70 reports

All LLM Security Agentic AI Industry News Research Supply Chain Prompt Injection First Look: Security Regulatory Jailbreaks Adversarial ML Data Poisoning Model Theft AI Security Tools Security Operations

Google DeepMind Releases AI Agent Attack Taxonomy

FIRST LOOK ATLAS OWASP HIGH ▲ 8.7 SecurityWeek Jun 25, 2026

Google DeepMind researchers have released a structured taxonomy categorising adversarial attacks against autonomous AI agents into six classes — content injection, semantic manipulation, cognitive state poisoning, behavioural control, systemic, and human-in-the-loop traps — formalising an emerging threat model for agentic AI systems. For defenders, this framework codifies attack paths that exploit the agent's inability to distinguish trusted instructions from attacker-controlled data ingested from web pages, emails, documents, and tool outputs. NIST evaluation data cited in the research shows malicious instruction injection succeeded in 57% of tested agent hijacking scenarios on average, underscoring that these are active, high-yield attack vectors rather than theoretical concerns.

First Look: Agentic AI SOC Systems Ship Autonomous Decision-Making at Machine Speed

FIRST LOOK ATLAS OWASP HIGH ▲ 7.8 SecurityWeek Jun 25, 2026

Agentic AI systems deployed in security operations and enterprise workflows are increasingly executing autonomous decisions at machine speed, using LLM-derived confidence regardless of context accuracy. The core security risk is that incomplete, poisoned, or manipulated context fed to these agents produces confidently wrong actions executed without human review. Defenders face a compounded threat: adversaries can now target the context layer—asset inventories, threat feeds, exposure data—to induce systematic misconfiguration or inaction at scale.

Dragos Launches EmberAI, an OT-Specific AI Platform

FIRST LOOK ATLAS OWASP HIGH ▲ 7.2 SecurityWeek Jun 24, 2026

Dragos has launched EmberAI, an AI module embedded within its OT security platform that allows analysts to query threat intelligence, asset data, and network activity in plain language, grounded in a decade of proprietary OT-specific data. The system introduces new attack surface considerations because it aggregates highly sensitive OT network telemetry, vulnerability data, and adversary intelligence into a single AI-queryable layer — making the platform itself a high-value target. Defenders must weigh the risks of prompt injection, over-reliance on AI-generated recommendations in safety-critical environments, and the intelligence value this consolidated dataset represents to nation-state adversaries.

LLM Role Confusion Attack Bypasses Safety at 61%

ATLAS OWASP HIGH ▲ 8.2 Simon Willison Jun 23, 2026

New research from Ye, Cui, and Hadfield-Menell demonstrates that LLMs prioritise the stylistic format of text over its structural role tags, enabling attackers to craft injected content that mimics internal reasoning blocks and bypasses safety guardrails. The study found attack success rates of 61% when injected text stylistically matched model-internal formats, dropping to just 10% after 'destyling'. The authors conclude that without genuine role perception in models, prompt injection defences will remain fundamentally reactive.

AWS Launches Bedrock AgentCore for Autonomous Payments

FIRST LOOK ATLAS OWASP HIGH ▲ 7.8 AWS Machine Learning Blog Jun 23, 2026

AWS has launched Amazon Bedrock AgentCore Payments, a managed infrastructure layer that enables AI agents to autonomously transact with external model providers and services using the x402 payment protocol, without human intervention. This capability introduces a new class of financial attack surface where compromised or manipulated agents can autonomously spend real funds, exfiltrate value, or be redirected to malicious service endpoints. Defenders must now treat agent payment credentials and spending budgets as first-class financial controls, on par with cloud IAM policies.

Bayer and Thoughtworks Ship PRINCE Agentic RAG Platform

FIRST LOOK ATLAS OWASP HIGH ▲ 7.2 HN AI Security Jun 22, 2026

Bayer AG and Thoughtworks have published a detailed case study on PRINCE, a production agentic RAG system combining multi-agent orchestration, Text-to-SQL, and human-in-the-loop workflows to answer complex pharmaceutical preclinical research questions and draft regulatory documents. The system's architecture — spanning intent clarification, planning, retrieval, reflection, and writing agents with access to decades of safety study data — introduces a broad attack surface including prompt injection across agent boundaries, SQL injection via natural language, and sensitive data exfiltration through compromised agent outputs. Defenders evaluating similar agentic platforms should treat each inter-agent handoff as a trust boundary requiring independent validation and focus on data leakage controls given the sensitivity of preclinical regulatory data.

Anthropic Launches Claude Code with Local Memory Layer

FIRST LOOK ATLAS OWASP MEDIUM ▲ 5.8 Anthropic (via HN) Jun 22, 2026

Recall is an open-source, fully-local memory layer for Anthropic's Claude Code that persists and summarises project context across coding sessions without sending data to external services. For defenders, the introduction of a persistent, file-based context store creates a new attack surface: a poisoned or tampered memory file can silently inject malicious instructions into every subsequent Claude Code session. Security teams should treat the local memory store as a trusted-input boundary and apply appropriate file-integrity and access controls.

OpenAI Ships GPT-5.5 Instant with Health Intelligence

FIRST LOOK ATLAS OWASP MEDIUM ▲ 5.8 OpenAI Blog Jun 21, 2026

OpenAI has upgraded ChatGPT's health and wellness response capabilities via GPT-5.5 Instant, incorporating stronger reasoning, physician-informed evaluations, and improved contextual understanding for medical queries. This expansion into high-stakes health guidance raises meaningful concerns for defenders, as improved fluency and authority in medical responses increases the risk of user overreliance and lowers the perceived threshold for trusting AI-generated health advice. Security and trust-safety teams should evaluate how this capability interacts with prompt injection, social engineering chains, and the broader risk of AI-mediated medical misinformation at scale.

Malware Uses Prompt Injection in JavaScript to Evade LLM Tools

ATLAS OWASP HIGH ▲ 8.2 Schneier on Security Jun 21, 2026

A malware developer has been observed embedding fake system instructions and policy-triggering content — including references to nuclear and biological weapons — inside JavaScript comment blocks to confuse or trigger refusal behaviour in LLM-powered security analysis pipelines. The technique does not affect code execution but is specifically designed to disrupt naive AI-first triage tools that feed raw file content to language models without isolating it as untrusted data. Traditional static analysis methods remain unaffected, but the approach signals an emerging class of anti-AI-analysis evasion techniques.

Enterprise Security Platforms Ship Autonomous Threat Response

FIRST LOOK ATLAS OWASP HIGH ▲ 7.2 The Hacker News Jun 21, 2026

A new class of agentic AI security platforms is emerging that autonomously correlates threat intelligence, validates controls, and prioritizes remediations across siloed enterprise security tooling — moving beyond assistive chatbot interfaces to continuous, multi-step autonomous action. This shift introduces significant new attack surface: an AI system with persistent access to live exposure data, security telemetry, and remediation workflows becomes a high-value target for adversarial manipulation. Defenders must assess trust boundaries, prompt injection risks, and the consequences of autonomous action taken on poisoned or manipulated inputs before deploying these systems.

GitHub Ships Data Analytics Agent Built on Copilot

FIRST LOOK ATLAS OWASP MEDIUM ▲ 6.8 GitHub Blog Jun 20, 2026

GitHub has published a detailed engineering account of how it built an internal data analytics agent using GitHub Copilot, exposing the architectural patterns — including natural language-to-SQL translation, autonomous tool invocation, and internal data access — that underpin such systems. For defenders, this blueprint highlights concrete risks around prompt injection into analytics pipelines, excessive agency over sensitive internal datasets, and the challenge of auditing LLM-generated queries before execution. Organisations adopting similar agentic analytics patterns should treat this as a reference threat model rather than a safe-to-copy architecture.

AutoGen Studio RCE: AutoJack Exploit Chain Targets Developers

ATLAS OWASP HIGH ▲ 9.1 The Hacker News Jun 20, 2026

Microsoft researchers disclosed AutoJack, an exploit chain targeting AutoGen Studio's MCP WebSocket endpoint that allows a single malicious web page to execute arbitrary commands on a developer's host machine via an AI browsing agent. The attack chains three distinct weaknesses — localhost trust bypass, missing authentication on MCP paths, and unsanitised command execution — requiring no credentials or user interaction beyond the agent loading the attacker's URL. While the vulnerable handler was not included in stable PyPI releases, it shipped in two pre-release builds that remain unyanked, leaving anyone who installed those versions exposed.

AWS Launches Amazon Bedrock AgentCore Harness

FIRST LOOK ATLAS OWASP HIGH ▲ 8.2 AWS Machine Learning Blog Jun 19, 2026

AWS has made Amazon Bedrock AgentCore Harness generally available, providing a managed abstraction layer that reduces agent deployment to two API calls while bundling sandboxed compute, persistent memory, tool gateway, browser access, identity management, and observability. For defenders, this dramatically lowers the barrier to deploying autonomous agents with filesystem access, shell execution, web browsing, and multi-provider model switching — compressing what was a weeks-long infrastructure project into minutes. Security teams face an expanded attack surface where prompt injection, tool abuse, cross-session memory poisoning, and supply chain risks through AWS-curated skill catalogs now arrive as a single, tightly integrated managed service rather than individually reviewable components.

AWS Launches Amazon Quick Autonomous Agents

FIRST LOOK ATLAS OWASP HIGH ▲ 8.2 AWS Machine Learning Blog Jun 18, 2026

AWS has shipped autonomous agents in Amazon Quick, an AI assistant that continuously executes tasks — including CRM updates, email drafting, and compliance monitoring — on behalf of users while connected to dozens of enterprise data sources and applications. This dramatically expands the attack surface for business-context compromise: a single successful prompt injection or account takeover can now translate into persistent, automated actions across an organisation's entire connected app ecosystem. Defenders must treat these agents as privileged service accounts with broad, continuous write-access, requiring dedicated monitoring, least-privilege scoping, and explicit human-in-the-loop gates for sensitive actions.

Google Launches Android 17 with Gemini Omni Integration

FIRST LOOK ATLAS OWASP HIGH ▲ 7.2 TechCrunch AI Jun 17, 2026

Android 17 embeds Gemini Omni and multiple AI models (Lyria 3, AudioLM) directly into OS-level functions including video editing, call handling, screen recording, and emergency detection, dramatically expanding the attack surface for AI-assisted exploitation on mobile endpoints. The deep integration of conversational AI with device sensors, media pipelines, and inter-app communication creates novel prompt injection and data exfiltration vectors that existing mobile threat defences were not designed to address. The simultaneous AirDrop interoperability expansion and cross-device Pixel Watch mirroring further widen the lateral movement surface across the Google hardware ecosystem.

Prompt-Injection

Stay ahead of the threat.