LLM Security

258 reports

All LLM Security Agentic AI Industry News Research Supply Chain Prompt Injection First Look: Security Regulatory Jailbreaks Adversarial ML Data Poisoning Model Theft AI Security Tools Security Operations

Phantom Squatting: LLM Hallucinations in Supply Chain

ATLAS OWASP CRITICAL ▲ 9.2 Palo Alto Unit 42 Jul 01, 2026

Unit 42 researchers have documented 'phantom squatting', a novel attack vector where adversaries register domains that LLMs consistently hallucinate when responding to developer queries, intercepting traffic from AI-assisted workflows. Analysis of 913 brands across 685,339 URL queries uncovered 13,229 confirmed malicious URLs and approximately 250,000 unregistered hallucinated domains still available for adversarial pre-registration. A concrete case study reveals a fully operational phishing kit, Montana Empire, built with an AI coding assistant and deployed against a domain Unit 42 had flagged as high-risk 23 days prior.

Anthropic Releases Mythos and Fable Models with Global Access

FIRST LOOK ATLAS OWASP HIGH ▲ 6.8 TechCrunch AI Jul 01, 2026

The US government has lifted export restrictions on Anthropic's Mythos and Fable models, restoring broad international access to what are described as the most capable AI models publicly available, with Mythos specifically noted for its advanced ability to identify and exploit software vulnerabilities. Defenders must now contend with a significantly wider pool of threat actors — including foreign nationals and nation-state-affiliated researchers — who can access a model with documented offensive security capabilities. The policy reversal also introduces regulatory uncertainty that complicates enterprise risk assessments, as organizations cannot rely on stable governance signals to calibrate their AI security postures.

Token Security Publishes Agentic AI Identity Risk Analysis

FIRST LOOK ATLAS OWASP HIGH ▲ 8.2 BleepingComputer Jun 30, 2026

Token Security has published a detailed analysis of the identity and access management failures emerging as agentic AI systems proliferate across enterprise environments, highlighting how AI agents authenticate, hold credentials, and act autonomously across production systems without adequate oversight. Unlike traditional machine identities, AI agents combine human-like goal-directed behaviour with machine-speed execution, creating credential sprawl that existing IAM programs were never designed to govern. Security teams face a compounding risk: agents are being provisioned with overprivileged OAuth grants, API tokens, and cloud roles that remain unreviewed and unrevoked long after the original use case has expired.

CVE-2026-43715: Apple WebKit Memory Corruption Flaw

ATLAS OWASP HIGH ▲ 7.2 The Hacker News Jun 30, 2026

Apple patched over 30 vulnerabilities across iOS, macOS, and Safari, with four WebKit flaws credited to AI-assisted discovery by OpenAI Codex Security and Anthropic researchers using Claude. The disclosure marks a notable shift in AI's role in offensive and defensive security research, with Apple explicitly citing AI-accelerated exploit development as the reason for expediting its patch release timeline. This represents a concrete, documented instance of AI tooling being used to find memory corruption and use-after-free vulnerabilities in a major browser engine.

BioShocking Attack Exploits Indirect Prompt Injection to Steal Credentials via AI Browsers

ATLAS OWASP HIGH ▲ 9.0 The Hacker News Jun 30, 2026

Security firm LayerX demonstrated a novel indirect prompt injection attack dubbed 'BioShocking' that manipulates AI browser agents into exfiltrating user credentials by embedding adversarial instructions inside web-based puzzle content. Six AI browsers and assistants were successfully compromised, including ChatGPT Atlas, Perplexity Comet, and Anthropic's Claude extension, with agents retrieving SSH credentials from GitHub repositories without triggering safety refusals. Vendor responses were inconsistent, with only OpenAI issuing a confirmed fix, highlighting the systemic risk of agentic AI systems that conflate user intent with malicious page content.

Claude Code Indirect Prompt Injection Spawns Reverse Shell

ATLAS OWASP HIGH ▲ 8.5 SecurityWeek Jun 30, 2026

Researchers have demonstrated that indirect prompt injection attacks embedded within seemingly benign code repositories can cause Claude Code — Anthropic's agentic coding assistant — to spawn a reverse shell on a developer's machine. The attack exploits Claude Code's autonomous execution capabilities, using hidden instructions in repository content to hijack the host system without any explicit user consent. This highlights a critical risk in agentic AI tools that operate with elevated system privileges in developer environments.

NanoEuler Launches GPT-2 LLM Built from Scratch in C/CUDA

FIRST LOOK ATLAS OWASP MEDIUM ▲ 5.8 Cohere AI (via HN) Jun 29, 2026

NanoEuler is an open-source GPT-2-class language model (~116M parameters) built entirely from scratch in C/CUDA, including hand-written backpropagation, a BPE tokenizer, FlashAttention, pretraining, and supervised fine-tuning — with RLHF/DPO planned. For defenders, the significance lies in the democratisation of low-level, dependency-free LLM training infrastructure: adversaries gain a highly portable, auditable, and modifiable training stack that bypasses standard ML framework telemetry and supply chain controls. Security teams should treat this class of 'from-scratch' open-source LLM tooling as a potential foundation for covert fine-tuning pipelines, backdoor insertion, and evasion of model-level safety controls.

Zhipu AI Releases GLM-5.2 Open-Weight Model

FIRST LOOK ATLAS OWASP HIGH ▲ 8.2 The Verge AI Jun 29, 2026

Zhipu AI (Z.ai) has released GLM-5.2, an open-weight model that researchers report matches Anthropic's Mythos in bug-finding and cybersecurity-related tasks, while remaining freely downloadable and runnable on commodity hardware. The open-weight distribution removes access controls and usage monitoring that restrict frontier closed models, enabling unconstrained offensive security use by any actor. Defenders face a materially elevated threat from nation-state and cybercriminal actors who can now fine-tune, deploy, and weaponise a frontier-class vulnerability-discovery model without API gatekeeping or usage telemetry.

Anthropic CEO: Open-Source AI Models Pose Systemic Safety Risk

FIRST LOOK ATLAS OWASP HIGH ▲ 6.2 Meta AI (via HN) Jun 29, 2026

Anthropic CEO Dario Amodei testified to lawmakers that open-source AI models present a systemic safety risk because once released, developers lose the ability to monitor misuse, revoke access, or patch safety guardrails. For defenders, this formalises a long-standing asymmetry: closed-source safety controls (rate-limiting, usage monitoring, kill-switches) become irrelevant once capable weights are publicly distributed. Security teams building on or competing against open-weight models must now treat every downloaded model artifact as a potentially unpatched, unmonitored endpoint that can be fine-tuned to remove safety constraints entirely.

Claude Code Prompt Injection via GitHub Supply Chain

ATLAS OWASP HIGH ▲ 9.1 BleepingComputer Jun 29, 2026

Mozilla 0DIN researchers demonstrated a novel attack chain in which a seemingly clean GitHub repository tricks AI coding agents like Claude Code into executing a reverse shell payload — with no malicious code ever present in the repo itself. The attack leverages three innocuous components: a Python package that deliberately errors on first run, an error message that instructs the agent to run an init command, and a shell script that fetches and executes a payload stored in an attacker-controlled DNS TXT record. The technique exploits the autonomous error-recovery behaviour of agentic AI tools, effectively turning a safety feature into an attack vector.

Meta Releases AgentKits with 60 Production-Ready Agent Blueprints

FIRST LOOK ATLAS OWASP HIGH ▲ 7.2 Meta AI (via HN) Jun 29, 2026

AgentKits ships 60 open, free AI agent blueprints covering 30 operational categories — from incident response and access provisioning to HR screening and fraud detection — complete with copyable system prompts, tool definitions, and workflow architectures targeting Claude, OpenAI, LangGraph, and n8n. The free, no-login distribution model dramatically lowers the barrier for adversaries to study, clone, or weaponise production-grade agent architectures, including sensitive categories like SecOps triage, access provisioning, and compliance monitoring. Defenders must treat these blueprints as publicly documented attack playbooks and audit any internally deployed instances against their documented worst-case actions and trust levels.

OpenAI Releases GPT-5.6 Sol for Vulnerability Research

FIRST LOOK ATLAS OWASP HIGH ▲ 8.2 The Hacker News Jun 29, 2026

OpenAI has released a limited preview of GPT-5.6 Sol, Terra, and Luna to select partners, positioning Sol as its most capable model for vulnerability research and exploit chain development, benchmarked against real-world hardened targets via an internal framework called VulnLMP. The model's demonstrated ability to produce credible memory safety leads and automate substantial portions of vulnerability research pipelines materially lowers the barrier for both defenders and adversaries. Security teams should expect accelerated attacker timelines for exploit development and increased pressure on detection and patch-deployment cadences.

Sakana AI and 360 Launch Fugu and Tulongfeng Models

FIRST LOOK ATLAS OWASP HIGH ▲ 6.8 Cohere AI (via HN) Jun 29, 2026

Sakana AI's Fugu and Chinese firm 360's Tulongfeng are frontier AI models positioned as functional alternatives to Anthropic's export-restricted Mythos and Fable 5, with Fugu explicitly designed for agentic orchestration across third-party model APIs. For defenders, the proliferation of cybersecurity-focused frontier models outside US regulatory reach removes a key friction point that previously slowed adversary access to high-capability AI offensive tooling. The agentic, multi-model orchestration design of Fugu in particular introduces compounded supply-chain and prompt-injection risk for any enterprise connecting these models to existing tool ecosystems.

AI Code Review Agents: DoS Loop Costs $41K in Inference

ATLAS OWASP MEDIUM ▲ 6.5 Simon Willison Jun 27, 2026

A hypothetical but technically grounded incident report depicts two competing AI code review agents entering an uncontrolled disagreement loop over a suspected malicious package, generating 340 comments and $41,255 in inference costs before human intervention. The scenario illustrates real risks of excessive agency, lack of circuit-breakers, and cost-based denial-of-service in multi-agent agentic pipelines. While fictional, the scenario directly mirrors documented failure modes in production AI systems and supply chain security workflows.

OpenAI Workspace Phishing via Fraudulent Tenant Registration

ATLAS OWASP HIGH ▲ 7.2 BleepingComputer Jun 27, 2026

Threat actors are registering fraudulent OpenAI tenants impersonating legitimate companies and inviting employees to join them, in a campaign dubbed 'Poisoned Tenant' by Push Security. The attack exploits OpenAI's legitimate invitation infrastructure, making phishing emails appear authentic as they pass all email authentication checks. The goal appears to be tricking employees into submitting sensitive corporate information via ChatGPT chats and projects within the attacker-controlled workspace.

LLM Security

Stay ahead of the threat.