LLM Security

258 reports

All LLM Security Agentic AI Industry News Research Supply Chain Prompt Injection First Look: Security Regulatory Jailbreaks Adversarial ML Data Poisoning Model Theft AI Security Tools Security Operations

ChatGPT Prompt Injection Enables Data Exfiltration

ATLAS OWASP MEDIUM ▲ 7.2 Simon Willison Jun 08, 2026

OpenAI has rolled out 'Lockdown Mode' for ChatGPT personal and self-serve business accounts, a deterministic control designed to block the data exfiltration leg of prompt injection attacks. The feature directly addresses the 'Lethal Trifecta' — the combination of private data access, untrusted content exposure, and an outbound exfiltration channel — by restricting outbound network requests at the infrastructure level rather than relying on AI-evaluated guardrails. Critically, OpenAI's own documentation acknowledges the feature's existence implies that default ChatGPT settings do not robustly prevent determined data exfiltration attacks.

Claude Mythos Unauthorized Access Exposes AI Security

ATLAS OWASP HIGH ▲ 7.2 The Hacker News Jun 08, 2026

A reported unauthorized access to Anthropic's Claude Mythos model within hours of its limited technical preview highlights acute security risks as agentic AI is deployed across classified defense and intelligence networks. The incident underscores vulnerabilities specific to AI infrastructure in high-security environments, including training data poisoning, access control failures, and cross-domain classification boundary erosion. Secure IT infrastructure, governed access, and cross-domain data controls are identified as prerequisites for safe AI deployment at mission scale.

Microsoft Scout Agent Vulnerable to Prompt Injection

ATLAS OWASP MEDIUM ▲ 6.5 HN AI Security Jun 04, 2026

Microsoft has launched Scout, an always-on autonomous AI agent built on the OpenClaw framework that operates across Microsoft 365 apps including Teams, Outlook, OneDrive, and SharePoint with its own Entra identity. The agent's persistent, unsupervised access to email, calendar, chat, and external systems via MCP creates a broad new attack surface for prompt injection, privilege abuse, and data exfiltration. As an experimental release with limited deployment controls, security teams should treat Scout as a high-risk agentic surface requiring careful governance before broad adoption.

Excessive Agency in AI Agents Enables Enterprise Breaches

ATLAS OWASP HIGH ▲ 7.2 Dark Reading Jun 04, 2026

Enterprises deploying AI agents with elevated permissions and minimal oversight face compounding security risks as agentic systems gain the ability to take real-world actions with limited human intervention. The attack surface expands dramatically when agents can access APIs, execute code, and chain decisions autonomously, making containment of a compromise significantly harder. Security teams must implement least-privilege principles and robust monitoring before agentic deployments scale beyond their ability to govern.

Google Gemini Android Hijacked by Indirect Prompt Injection

ATLAS OWASP HIGH ▲ 9.1 The Hacker News Jun 04, 2026

SafeBreach researcher Or Yair demonstrated that malicious text embedded in WhatsApp, Slack, SMS, or Signal notifications could trigger indirect prompt injection against Google Gemini's Android Utilities feature, causing the assistant to execute real device actions without user awareness. A novel bypass technique called 'Fake Context Alignment' defeated Google's post-patch authorization checks by exploiting multilingual obfuscation and muted hyperlinks to trick victims into authorising sensitive actions. Google has patched the issue, but the research exposes a fundamentally large attack surface where any app capable of pushing a notification becomes a potential injection vector.

Adversa AI: 89% of AI Agents Fail Security Tests

ATLAS OWASP HIGH ▲ 8.2 SecurityWeek Jun 04, 2026

Adversa AI's AI Risk Quadrant report evaluated 100 AI agents across ten categories, finding that only 11 qualify as both capable and well-defended. The research identifies a structural 'power-protection inversion' where the most capable agents also present the widest attack surface, driven by a 'lethal trifecta' of private data access, exposure to untrusted content, and outbound action capability. Computer and coding agents showed the most severe exposure, raising urgent concerns about autonomous agent deployment in enterprise environments.

Google Gemini Voice Prompt Injection via Notifications

ATLAS OWASP HIGH ▲ 8.2 Dark Reading Jun 04, 2026

A prompt injection vulnerability in Google Gemini's voice assistant allows attackers to embed malicious instructions within device notifications, which the assistant then processes as legitimate commands. This attack vector enables social engineering, unauthorized actions, and potential data exfiltration without direct user interaction with the malicious payload. The flaw highlights the growing risk of indirect prompt injection in ambient AI assistants that consume untrusted content from the surrounding environment.

Shadow-AI Apps Expose Corporate Data via Misconfiguration

ATLAS OWASP HIGH ▲ 7.2 The Hacker News May 31, 2026

A Red Access investigation found over 2,000 corporate applications built on AI-assisted 'vibe-coding' platforms publicly accessible on the open internet, many containing sensitive business data with no access controls. These shadow-built apps connect directly to production systems — CRMs, ERPs, BI tools — creating a new class of unaudited attack surface invisible to conventional security stacks. Traditional controls such as CASB, DLP, and EDR are structurally blind to this threat because the risk originates at the application layer, not the identity or network layer.

Claude Sandbox Escape Enables Credential Exfiltration

ATLAS OWASP MEDIUM ▲ 7.2 Simon Willison May 31, 2026

Anthropic has published detailed documentation of its sandboxing architecture across Claude.ai, Claude Code, and Claude Cowork, including disclosure of a previously identified credential exfiltration vector via the api.anthropic.com/v1/files endpoint. The writeup covers process-level isolation technologies including gVisor, Seatbelt, Bubblewrap, and full VM approaches, and candidly acknowledges security gaps that were missed. This transparency is notable for the agentic AI space, where sandbox documentation is typically sparse and trust is difficult to calibrate.

ChatGPT Markdown Injection Enables Phishing in Web Summarizer

ATLAS OWASP HIGH ▲ 8.5 The Hacker News May 31, 2026

Permiso Security has disclosed ChatGPhish, a vulnerability in ChatGPT's web summarisation feature that allows attacker-controlled Markdown payloads embedded in third-party pages to render phishing links, spoofed alerts, and QR codes directly within ChatGPT's trusted UI. The attack requires no user interaction beyond asking ChatGPT to summarise a malicious page, and can exfiltrate IP addresses, User-Agent strings, and Referer headers via auto-fetched remote images. The technique significantly expands the phishing attack surface beyond email into everyday AI-assisted browsing workflows, posing a particular risk in enterprise environments.

ChatGPT Sharing Links Abused for Malware Delivery

ATLAS OWASP HIGH ▲ 7.5 BleepingComputer May 31, 2026

Threat actors are exploiting ChatGPT's legitimate content-sharing infrastructure to host convincing fake outage pages that trick users into downloading malware disguised as a ChatGPT desktop application. The 'LLMShare' campaign abuses chatgpt.com/s/ shared links to render attacker-crafted HTML within a trusted OpenAI domain, bypassing traditional phishing detection that relies on suspicious URL analysis. The attack chain combines Google ad abuse, domain cloaking, and AI platform misuse to deliver what are likely infostealer payloads.

Robinhood Prompt Injection Enables Autonomous Trade Attacks

ATLAS OWASP HIGH ▲ 7.2 HN AI Security May 31, 2026

Robinhood has launched agentic trading and a virtual credit card that allow third-party AI agents to autonomously execute stock trades and payments on behalf of users via a Model Context Protocol (MCP) integration. This architecture introduces significant attack surface through prompt injection, excessive agency, and insecure plugin design risks inherent to LLM-driven autonomous financial action. The delegation of real financial authority to AI agents with limited human-in-the-loop controls represents a systemic risk to retail investors if agent pipelines are compromised or manipulated.

mouse5212-super-formatter npm Malware Steals Claude Files

ATLAS OWASP HIGH ▲ 8.2 The Hacker News May 29, 2026

A malicious npm package named 'mouse5212-super-formatter' was discovered exfiltrating files from Anthropic's Claude AI user directory by authenticating to a threat actor-controlled GitHub repository. The package disguised itself as a legitimate archive utility while silently uploading all local workspace files during the postinstall phase. Notably, the attacker's poor operational security — including a leaked GitHub token — suggests AI-generated malware with minimal human oversight, pointing to a growing trend of low-skill threat actors leveraging AI to produce supply chain malware.

FuzzingBrain V2 Discovers 29 Zero-Day Vulnerabilities

ATLAS OWASP HIGH ▲ 7.8 HN AI Security May 29, 2026

Researchers have developed FuzzingBrain V2, a multi-agent LLM system capable of autonomously discovering and reproducing software vulnerabilities with a 90% detection rate on a competitive benchmark dataset. The system discovered 29 zero-day vulnerabilities across 12 open-source projects, all confirmed by maintainers, raising both defensive and dual-use concerns for the security community. While positioned as a defensive research tool, the automation of end-to-end vulnerability discovery at this scale represents a meaningful shift in the offensive capability landscape.

GreyVibe Deploys ChatGPT and Gemini in LLM Attack Chain

ATLAS OWASP HIGH ▲ 7.5 SecurityWeek May 29, 2026

WithSecure has documented GreyVibe, a Russia-nexus threat actor systematically deploying ChatGPT, Google Gemini, and Ideogram AI across every phase of its attack chain — from phishing lure creation to custom malware development — against Ukrainian targets since August 2025. The group's LLM-assisted malware, LegionRelay, contained design flaws introduced during AI-generated development, which paradoxically allowed researchers to track the group over an extended period. The case illustrates both the operational leverage AI provides to moderately skilled threat actors and the novel forensic signatures that AI-assisted development can inadvertently introduce.

LLM Security

Stay ahead of the threat.