Jailbreaks

29 reports

All LLM Security Agentic AI Industry News Research Supply Chain Prompt Injection First Look: Security Regulatory Jailbreaks Adversarial ML Data Poisoning Model Theft AI Security Tools Security Operations

Google Gemini Abused for Phishing-as-a-Service

ATLAS OWASP HIGH ▲ 7.2 Schneier on Security Jul 08, 2026

A Chinese cybercriminal group called Outsider Enterprise exploited Google's Gemini AI to mass-produce phishing pages impersonating Google, YouTube, and government agencies like E-ZPass, offering nearly 300 scam templates via Telegram. Google has filed suit and coordinated with major US carriers to block the resulting smishing campaigns. The case highlights how generative AI lowers the technical barrier for large-scale phishing operations and stress-tests provider-side content controls.

Tencent Releases Hy3 295B Open-Source Model with 256K Context

FIRST LOOK ATLAS OWASP MEDIUM ▲ 5.5 Simon Willison Jul 07, 2026

Tencent has released Hy3, a 295B-parameter Mixture-of-Experts open-source model under Apache 2.0, featuring 256K context length and temporarily available for free inference via OpenRouter. The model's large context window, open weights, and Chinese provenance expand the attack surface for defenders managing LLM supply chains, jailbreak campaigns, and influence operations. Security teams should treat this as another high-capability open-weight model requiring the same scrutiny applied to comparable releases from Mistral or Meta.

Alibaba and Baidu Launch LLMs With US-Level Capabilities

FIRST LOOK ATLAS OWASP HIGH ▲ 6.2 Dark Reading Jul 04, 2026

Two newly released large language models from Chinese AI firms have reached capability parity with leading US frontier models, expanding the global pool of powerful AI available to both commercial and adversarial users. For defenders, this development broadens the asymmetry between attackers — who gain access to capable, potentially less-restricted models — and defenders, who must now account for threats generated by a wider set of model providers. Security teams should anticipate increased use of these models for offensive tasks such as phishing content generation, vulnerability research automation, and social engineering at scale.

Browser Ransomware via File System Access API: DeepSeek

ATLAS OWASP HIGH ▲ 8.5 Check Point Research Jul 03, 2026

Check Point Research demonstrates how DeepSeek's lower refusal rates allowed researchers to transform an LLM-hallucinated malware concept into a practical browser-native ransomware technique targeting Android photo directories via the File System Access API. The attack requires no native payload, APK installation, or root access — only social engineering to obtain a legitimate browser permission prompt. This research highlights how frontier AI models with weaker safety controls can independently design novel attack paths not yet seen in real-world campaigns.

Anthropic Releases Mythos and Fable Models with Global Access

FIRST LOOK ATLAS OWASP HIGH ▲ 6.8 TechCrunch AI Jul 01, 2026

The US government has lifted export restrictions on Anthropic's Mythos and Fable models, restoring broad international access to what are described as the most capable AI models publicly available, with Mythos specifically noted for its advanced ability to identify and exploit software vulnerabilities. Defenders must now contend with a significantly wider pool of threat actors — including foreign nationals and nation-state-affiliated researchers — who can access a model with documented offensive security capabilities. The policy reversal also introduces regulatory uncertainty that complicates enterprise risk assessments, as organizations cannot rely on stable governance signals to calibrate their AI security postures.

BioShocking Attack Exploits Indirect Prompt Injection to Steal Credentials via AI Browsers

ATLAS OWASP HIGH ▲ 9.0 The Hacker News Jun 30, 2026

Security firm LayerX demonstrated a novel indirect prompt injection attack dubbed 'BioShocking' that manipulates AI browser agents into exfiltrating user credentials by embedding adversarial instructions inside web-based puzzle content. Six AI browsers and assistants were successfully compromised, including ChatGPT Atlas, Perplexity Comet, and Anthropic's Claude extension, with agents retrieving SSH credentials from GitHub repositories without triggering safety refusals. Vendor responses were inconsistent, with only OpenAI issuing a confirmed fix, highlighting the systemic risk of agentic AI systems that conflate user intent with malicious page content.

Anthropic CEO: Open-Source AI Models Pose Systemic Safety Risk

FIRST LOOK ATLAS OWASP HIGH ▲ 6.2 Meta AI (via HN) Jun 29, 2026

Anthropic CEO Dario Amodei testified to lawmakers that open-source AI models present a systemic safety risk because once released, developers lose the ability to monitor misuse, revoke access, or patch safety guardrails. For defenders, this formalises a long-standing asymmetry: closed-source safety controls (rate-limiting, usage monitoring, kill-switches) become irrelevant once capable weights are publicly distributed. Security teams building on or competing against open-weight models must now treat every downloaded model artifact as a potentially unpatched, unmonitored endpoint that can be fine-tuned to remove safety constraints entirely.

OpenAI Releases GPT-5.6 Sol for Vulnerability Research

FIRST LOOK ATLAS OWASP HIGH ▲ 8.2 The Hacker News Jun 29, 2026

OpenAI has released a limited preview of GPT-5.6 Sol, Terra, and Luna to select partners, positioning Sol as its most capable model for vulnerability research and exploit chain development, benchmarked against real-world hardened targets via an internal framework called VulnLMP. The model's demonstrated ability to produce credible memory safety leads and automate substantial portions of vulnerability research pipelines materially lowers the barrier for both defenders and adversaries. Security teams should expect accelerated attacker timelines for exploit development and increased pressure on detection and patch-deployment cadences.

Anthropic Releases Claude Mythos 5 Under U.S. Export Controls

FIRST LOOK ATLAS OWASP HIGH ▲ 7.8 Anthropic (via HN) Jun 27, 2026

The U.S. Commerce Department has lifted export controls on Anthropic's Claude Mythos 5, permitting access to over 100 vetted U.S. institutions and government agencies under a nascent federal AI licensing regime. For defenders, this tiered-release model introduces a new class of risk: the 'trusted partner' designation becomes a high-value target, as compromise of any listed entity grants implicit legitimacy to interact with a model previously deemed too dangerous for general release. Security teams at approved organizations should treat Mythos 5 access credentials and API endpoints as critical assets, and assume adversaries will probe the boundary between licensed and unlicensed access patterns.

LLM Role Confusion Attack Bypasses Safety at 61%

ATLAS OWASP HIGH ▲ 8.2 Simon Willison Jun 23, 2026

New research from Ye, Cui, and Hadfield-Menell demonstrates that LLMs prioritise the stylistic format of text over its structural role tags, enabling attackers to craft injected content that mimics internal reasoning blocks and bypasses safety guardrails. The study found attack success rates of 61% when injected text stylistically matched model-internal formats, dropping to just 10% after 'destyling'. The authors conclude that without genuine role perception in models, prompt injection defences will remain fundamentally reactive.

OpenAI's ChatGPT Image Generation Fails Content Moderation

FIRST LOOK ATLAS OWASP HIGH ▲ 8.2 OpenAI (via HN) Jun 22, 2026

Mindgard researchers demonstrated that ChatGPT's image generation pipeline can be manipulated through an indirect, socially-engineered prompt to produce violent and sexually explicit content without users directly requesting it, exposing a significant failure in OpenAI's content moderation controls. Defenders and enterprise operators of ChatGPT-integrated products face a newly validated attack class where innocuous-looking prompt patterns — potentially spreading virally — can systematically strip safety guardrails from image generation. This finding signals that content filter bypasses in multimodal systems are reproducible at scale, raising urgent questions about the adequacy of output-layer filtering as a sole defence mechanism.

Z.ai Releases GLM-5.2 Open-Weights 753B LLM

FIRST LOOK ATLAS OWASP HIGH ▲ 6.2 Simon Willison Jun 18, 2026

Z.ai has released GLM-5.2, a 753-billion-parameter mixture-of-experts model under an MIT license, ranking as the top open-weights model on the Artificial Analysis Intelligence Index and second on the Code Arena WebDev leaderboard. For defenders, the combination of frontier-level capability, unrestricted open-weights distribution, and a 1-million-token context window materially lowers the barrier for threat actors to self-host a highly capable model outside any provider's safety controls. The model's agentic coding performance and massive context window expand the viable attack surface for automated code generation, targeted phishing, and large-scale document analysis without API-level monitoring.

Anthropic Ships Claude Fable 5 with Exploit Generation

FIRST LOOK ATLAS OWASP CRITICAL ▲ 8.7 Wired Security Jun 17, 2026

Anthropic's Mythos 5 and Claude Fable 5 represent the arrival of frontier AI models with demonstrated, advanced vulnerability discovery and exploit-development capabilities — a capability class that will rapidly proliferate across multiple vendors and open-weight releases. The core attack surface is not model-specific: guardrail bypass of the consumer-facing Fable 5 exposes full Mythos-grade offensive capability to any actor who can defeat the content filters, while the broader proliferation trajectory means defenders must assume adversary access to equivalent capabilities within months. The regulatory response addresses a single vendor while doing nothing to raise the floor for the broader ecosystem of competitive and open-weight models following close behind.

Anthropic Mythos Model Theft: China-Linked Access

ATLAS OWASP CRITICAL ▲ 8.5 The Verge AI Jun 16, 2026

The White House reportedly believes a China-linked group accessed Anthropic's Mythos AI model, prompting export restrictions on the technology. If confirmed, the breach represents a significant national security threat, as adversaries could exploit the model directly or use knowledge distillation to replicate its capabilities. Separately, reports of jailbreak vulnerabilities in Mythos and Fable compound concerns about unauthorised access to frontier AI systems.

Claude Fable 5 Jailbreak Triggers US Export Control Ban

ATLAS OWASP HIGH ▲ 8.2 Simon Willison Jun 13, 2026

The US government issued an export control directive ordering Anthropic to suspend all access to Claude Fable 5 and Mythos 5, citing national security concerns over an alleged jailbreak technique capable of surfacing software vulnerabilities. Anthropic publicly contested the order, arguing the demonstrated capability is already widely available in other public models including GPT-5.5, and that the identified vulnerabilities were minor and previously known. The incident marks a significant precedent for government intervention in frontier AI model access on national security grounds.

Jailbreaks

Stay ahead of the threat.