LLM Security

258 reports

All LLM Security Agentic AI Industry News Research Supply Chain Prompt Injection First Look: Security Regulatory Jailbreaks Adversarial ML Data Poisoning Model Theft AI Security Tools Security Operations

TeamPCP Steals Mistral AI Source Code via Supply Chain

ATLAS OWASP HIGH ▲ 8.5 BleepingComputer May 15, 2026

The TeamPCP threat group has compromised Mistral AI's codebase management system via the Shai-Hulud software supply chain attack, stealing approximately 5GB of internal repositories covering training, fine-tuning, benchmarking, and inference pipelines. The hackers are demanding $25,000 for nearly 450 repositories or threatening to leak them publicly within a week. Mistral AI confirmed the breach but stated that core repositories, hosted services, managed user data, and research environments were not affected.

Sweet Security Launches Sweet Attack Agentic AI Red Teaming

ATLAS OWASP MEDIUM ▲ 7.2 SecurityWeek May 14, 2026

Sweet Security has launched 'Sweet Attack', a continuous agentic AI red teaming platform designed to counter the growing asymmetry between AI-assisted attackers and human defenders — a tipping point the industry has termed the 'Mythos Moment'. The platform differentiates itself by grounding frontier model reasoning in live runtime telemetry from each customer's own environment, including topology, identity paths, and unencrypted Layer 7 exposure, to identify genuinely exploitable attack chains rather than theoretical ones. The development signals a broader industry shift toward autonomous, environment-aware AI agents as a necessary component of modern security operations.

AI Agents Generate Custom Malware in Mexico, Brazil

ATLAS OWASP HIGH ▲ 7.5 Dark Reading May 14, 2026

Two threat campaigns targeting organisations in Mexico and Brazil have leveraged AI agents to dynamically generate customised hacking tools, marking a notable escalation in automated, AI-assisted cyberattacks. The use of AI agents for on-the-fly tool generation lowers the technical barrier for attackers and accelerates the attack cycle. This represents a concrete, in-the-wild demonstration of agentic AI being exploited as an offensive capability.

GPT-5.5 and Claude Mythos Lower Barriers to Offensive AI

ATLAS OWASP HIGH ▲ 7.2 Schneier on Security May 14, 2026

The UK AI Security Institute has evaluated GPT-5.5 and found it comparable to Claude Mythos in identifying security vulnerabilities, with both models now generally available to the public. This parity raises serious concerns about the lowered barrier to entry for offensive cyber operations, as adversaries can leverage widely accessible models for vulnerability research. Commentary from security experts highlights that LLM-based vulnerability discovery is constrained to known attack patterns, but the existence of jailbreaks means guardrails provide only partial mitigation.

Microsoft MDASH Discovers 16 Windows RCE Flaws

ATLAS OWASP HIGH ▲ 7.8 The Hacker News May 14, 2026

Microsoft has disclosed MDASH, a multi-model agentic AI scanning system that autonomously discovered 16 vulnerabilities patched in May 2026's Patch Tuesday, including two critical RCE flaws. The system orchestrates over 100 specialised AI agents in a structured pipeline covering auditing, debating, and proof-of-exploitability stages. MDASH represents a significant shift in how AI is being deployed offensively and defensively within the vulnerability research lifecycle, with direct implications for how agentic AI systems are trusted, scoped, and governed.

OpenAI Daybreak Vulnerability Detection Enables LLM Jailbreak

ATLAS OWASP MEDIUM ▲ 7.2 The Hacker News May 13, 2026

OpenAI has launched Daybreak, an AI-powered cybersecurity platform combining GPT-5.5 variants and Codex Security to automate vulnerability detection, threat modelling, and patch validation for enterprise codebases. The initiative introduces a tiered model access structure — including a permissive 'GPT-5.5-Cyber' for red teaming — raising questions about dual-use risk and model misuse if access controls are circumvented. The rollout also contextualises a broader industry tension: AI is accelerating vulnerability discovery faster than defenders can remediate, contributing to triage fatigue and hallucinated bug reports.

Excessive Agency in AI Agents: Tool Access Control Gaps

ATLAS OWASP LOW ▲ 6.2 HN AI Security May 13, 2026

Statewright is an open-source framework that enforces state machine constraints on AI agents, restricting which tools agents can invoke during each phase of a workflow. The project directly addresses the Excessive Agency problem, where AI agents operating with broad, unconstrained tool access can take unintended or harmful actions. While a defensive development rather than a threat disclosure, it signals growing practitioner awareness of agentic AI risk and offers a concrete mitigation pattern for teams deploying coding agents like Claude Code, Codex, or Cursor.

CVE-2026-45321: Supply Chain Worm Targets Mistral AI

ATLAS OWASP CRITICAL ▲ 9.2 The Hacker News May 13, 2026

The TeamPCP threat actor has executed a broad supply chain campaign dubbed Mini Shai-Hulud, injecting credential-stealing malware into npm and PyPI packages from major AI and developer tooling ecosystems including Mistral AI, Guardrails AI, and TanStack. The malware profiles execution environments, exfiltrates cloud, CI, and AI tool credentials, and establishes persistence inside Claude Code and VS Code IDEs. The TanStack compromise alone affected 42 packages and 84 versions, exploiting a chained GitHub Actions attack to inject malicious payloads without stealing npm tokens directly.

AI-Powered Exploit Development by Threat Actors

ATLAS OWASP HIGH ▲ 7.2 Dark Reading May 12, 2026

Threat actors are now actively deploying large language models to accelerate exploit development and automate complex cyberattack workflows, marking a significant evolution in adversarial tooling. This shift lowers the technical barrier for sophisticated attack execution, enabling less-skilled actors to produce functional exploits at scale. The trend signals a structural change in the offensive threat landscape, with AI acting as a force multiplier for adversaries.

PromptSpy Zero-Day: AI-Generated Malware for Mass Exploitation

ATLAS OWASP CRITICAL ▲ 9.2 Mandiant Blog May 12, 2026

Google's Threat Intelligence Group (GTIG) has identified, for the first time, a criminal threat actor using a zero-day exploit believed to have been AI-generated, intended for mass exploitation before proactive counter-discovery intervened. The report also documents AI-augmented malware development, autonomous attack orchestration via AI-enabled malware (PROMPTSPY), and obfuscated LLM access pipelines used by adversaries to bypass usage controls. Nation-state actors from China and North Korea are actively pursuing AI-assisted vulnerability discovery, marking a significant escalation in adversarial AI capability.

AI-Generated Zero-Day: 2FA Bypass in Web Admin Tool

ATLAS OWASP CRITICAL ▲ 9.2 The Hacker News May 12, 2026

Google's Threat Intelligence Group has confirmed the first known instance of a threat actor using an AI model to discover and weaponize a zero-day vulnerability — a 2FA bypass in a popular open-source web administration tool. The exploit, delivered via a Python script bearing hallmarks of LLM-generated code (including hallucinated CVSS scores and structured docstrings), was designed for mass exploitation. This marks a significant inflection point in the offensive AI threat landscape, demonstrating that AI-assisted vulnerability discovery and weaponization has moved from theoretical risk to confirmed operational reality.

Steganography in LLMs Enables Covert Data Exfiltration

ATLAS OWASP MEDIUM ▲ 6.5 Schneier on Security May 12, 2026

Research highlighted by Bruce Schneier confirms that LLMs are highly effective at embedding hidden messages within seemingly normal text, a technique known as text-in-text steganography. This capability raises significant concerns for covert communications, data exfiltration, and the evasion of AI content moderation systems. Even small models with ~4 billion parameters demonstrate robust encoding and decoding of obfuscated language, lowering the barrier for adversarial misuse.

Typosquatted OpenAI Repo Delivers Rust Infostealer to 244K Users

ATLAS OWASP CRITICAL ▲ 8.5 The Hacker News May 11, 2026

A malicious Hugging Face repository impersonated OpenAI's legitimate Privacy Filter model, cloning its description verbatim to gain credibility and reach the platform's trending list with 244,000 downloads. The repository delivered a multi-stage attack chain culminating in a Rust-based information stealer targeting browser credentials, cryptocurrency wallets, and Discord data on Windows machines. The attack leveraged a dead-drop resolver pattern via a public JSON paste service, allowing operators to swap payloads without modifying the repository itself.

Hugging Face Supply Chain: Fake OpenAI Infostealer Hits 244K

ATLAS OWASP HIGH ▲ 8.2 BleepingComputer May 10, 2026

A malicious Hugging Face repository impersonating OpenAI's 'Privacy Filter' project reached #1 on the platform's trending list and accumulated 244,000 downloads before removal, delivering a multi-stage infostealer to Windows users. The attack chain used a disguised Python loader to execute PowerShell commands, ultimately deploying a Rust-based payload capable of harvesting browser credentials, crypto wallets, SSH/VPN configs, and screenshots. The campaign highlights the growing risk of AI/ML supply chain attacks through trusted model-sharing platforms.

Claude Chrome Extension Prompt Injection Enables Agent Takeover

ATLAS OWASP HIGH ▲ 9.1 SecurityWeek May 09, 2026

A vulnerability dubbed ClaudeBleed in Anthropic's Claude Chrome extension allows any browser extension to inject arbitrary prompts into the Claude AI agent by exploiting lax permission checks and improper trust validation. Attackers can bypass user confirmation protections via DOM manipulation and repeated message forging, enabling full agent takeover for information theft or unauthorized actions. The flaw effectively breaks Chrome's extension security model and exposes users running Claude's agentic capabilities to third-party extension compromise.

LLM Security

Stay ahead of the threat.