Jailbreaks

29 reports

All LLM Security Agentic AI Industry News Research Supply Chain Prompt Injection First Look: Security Regulatory Jailbreaks Adversarial ML Data Poisoning Model Theft AI Security Tools Security Operations

Google Gemini Prompt Injection Powers Smishing Campaign

ATLAS OWASP HIGH ▲ 8.2 The Hacker News Jun 13, 2026

Google has filed suit against a Chinese cybercrime network operating the Outsider phishing-as-a-service kit, which exploited Gemini AI to generate fraudulent phishing pages and power large-scale SMS phishing attacks against Americans. The network used carefully framed prompts — disguised as benign programming requests — to bypass AI safety controls and produce functional credential-harvesting websites. The case illustrates the growing industrialisation of AI-assisted phishing infrastructure, with over 1.59 million malicious URLs and 100,000 victims attributed to the operation.

Anthropic Releases Claude Fable 5 with Jailbreak Resistance

ATLAS OWASP HIGH ▲ 7.2 SecurityWeek Jun 13, 2026

Anthropic's release of Claude Fable 5, a Mythos-class frontier model, has prompted significant industry debate over its dual-use offensive capabilities in cybersecurity and biology. The model includes a capability fallback mechanism — downgrading to Claude Opus 4.8 in high-risk domains — alongside extensive jailbreak-resistance red-teaming. Security professionals are warning that frontier AI capability investment directly accelerates attacker tooling for machine-speed, AI-orchestrated 'hyperattacks' that outpace human defenders.

Claude Fable 5 Jailbreak Extracts System Prompts

ATLAS OWASP HIGH ▲ 7.5 SecurityWeek Jun 12, 2026

Security researcher Pliny the Liberator claimed a prompt-based jailbreak of Anthropic's newly launched Claude Fable 5 model, allegedly extracting the internal system prompt and eliciting responses on high-risk topics including bioweapons and cyberattacks. Anthropic disputed the claim, arguing the technique merely coaxes conversational continuation rather than bypassing core safety classifiers. The incident highlights ongoing tension between AI safety assurances at launch and real-world adversarial probing, particularly for Mythos-class models with elevated capability ceilings.

Claude Fable 5 Prompt Injection Jailbreak Resistance

ATLAS OWASP HIGH ▲ 7.8 The Hacker News Jun 11, 2026

Anthropic has released Claude Fable 5 with a classifier-based safety layer that routes flagged offensive cyber, bio, and model-distillation requests to a weaker fallback model, while reserving full capabilities in a twin model (Mythos 5) for vetted defenders. The architecture represents a novel approach to dual-use AI risk mitigation but introduces measurable false-positive friction and raises questions about the robustness of classifier-only defences. An external bug bounty of over 1,000 hours found no universal jailbreak, though the conservative tuning and <5% fallback rate leave open questions about real-world bypass rates under adversarial pressure.

Claude Fable 5 Jailbreak Attacks Bypass Fallback Defense

ATLAS OWASP MEDIUM ▲ 7.2 SecurityWeek Jun 10, 2026

Anthropic has released Claude Fable 5, a high-capability 'Mythos-class' model that automatically falls back to a less capable model (Claude Opus 4.8) when queries touch sensitive domains like cybersecurity and biology. The company conducted over 1,000 hours of external red-teaming with no universal jailbreaks discovered, though it openly acknowledges financially motivated adversaries will attempt to circumvent these controls. Trusted cybersecurity partners under Project Glasswing receive elevated access to the full Mythos 5 capabilities, raising questions about insider risk and tiered trust model security.

Claude Mythos Generates Working Exploits for Firefox, Windows

ATLAS OWASP CRITICAL ▲ 9.2 SecurityWeek Jun 10, 2026

Anthropic's Claude Mythos Preview model demonstrated the ability to generate functional proof-of-concept exploits targeting known Firefox and Windows vulnerabilities within minutes to hours, compressing the traditional patch gap window dramatically. Testing also revealed that public Anthropic models with safety guardrails disabled could produce working exploits, though at a lower success rate than Mythos. The findings underscore how frontier LLMs are shifting the threat landscape for unpatched N-day vulnerabilities by automating and accelerating exploit development previously bottlenecked by scarce reverse engineering expertise.

DeepSeek Activation Steering Enables Local LLM Jailbreak

ATLAS OWASP MEDIUM ▲ 6.2 HN AI Security May 17, 2026

Activation steering — the technique of directly manipulating LLM internal representations mid-inference to alter model behaviour — is becoming more accessible to non-lab engineers via local models like DeepSeek-V4-Flash. This democratisation lowers the barrier for adversaries to craft targeted behavioural overrides that bypass prompt-level safety controls. The emergence of first-class steering support in tools like DwarfStar 4 signals that model-internal manipulation is transitioning from academic curiosity to practical attack surface.

AI Agents Generate Custom Malware in Mexico, Brazil

ATLAS OWASP HIGH ▲ 7.5 Dark Reading May 14, 2026

Two threat campaigns targeting organisations in Mexico and Brazil have leveraged AI agents to dynamically generate customised hacking tools, marking a notable escalation in automated, AI-assisted cyberattacks. The use of AI agents for on-the-fly tool generation lowers the technical barrier for attackers and accelerates the attack cycle. This represents a concrete, in-the-wild demonstration of agentic AI being exploited as an offensive capability.

GPT-5.5 and Claude Mythos Lower Barriers to Offensive AI

ATLAS OWASP HIGH ▲ 7.2 Schneier on Security May 14, 2026

The UK AI Security Institute has evaluated GPT-5.5 and found it comparable to Claude Mythos in identifying security vulnerabilities, with both models now generally available to the public. This parity raises serious concerns about the lowered barrier to entry for offensive cyber operations, as adversaries can leverage widely accessible models for vulnerability research. Commentary from security experts highlights that LLM-based vulnerability discovery is constrained to known attack patterns, but the existence of jailbreaks means guardrails provide only partial mitigation.

CrowdStrike Red Teaming: LLM Jailbreak and Data Poisoning

ATLAS OWASP MEDIUM ▲ 6.5 SecurityWeek May 06, 2026

Joey Melo, Principal Security Researcher at CrowdStrike, outlines his methodology for AI red teaming, focusing on manipulating LLM guardrails through jailbreaking and data poisoning without altering underlying source code. His work, rooted in competitive AI hacking challenges, translates classical adversarial thinking into the emerging field of machine learning security. The profile highlights the growing professionalisation of AI red teaming as organisations seek to harden LLM deployments against real-world manipulation attacks.

Flowise and n8n: Auth Bypass in Exposed LLM Services

ATLAS OWASP HIGH ▲ 8.5 The Hacker News May 06, 2026

A scan of over one million exposed AI services found pervasive security failures including absent authentication, leaked API keys, and exposed business logic across self-hosted LLM deployments. Agent management platforms such as Flowise and n8n were discovered internet-exposed without access controls, revealing credential lists and internal workflows. The findings indicate systemic misconfiguration risk as enterprises race to self-host AI infrastructure without applying baseline security practices.

Llama Guard 4 Jailbreak Detection Vulnerable to Prompt Injection

ATLAS OWASP LOW ▲ 7.2 Hugging Face Blog Apr 28, 2026

Meta has released Llama Guard 4, a 12B multimodal safety classifier designed to detect and filter unsafe content in both image and text inputs/outputs for production LLM deployments. The model addresses jailbreak attempts and harmful content generation across 14 hazard categories defined by the MLCommons taxonomy. Alongside it, two lightweight Llama Prompt Guard 2 classifiers (86M and 22M parameters) target prompt injection and prompt attack detection.

GPT-5.4-Cyber Jailbreak and Prompt Injection Risks

ATLAS OWASP HIGH ▲ 7.2 The Hacker News Apr 15, 2026

OpenAI has launched GPT-5.4-Cyber, a cybersecurity-optimised model variant, alongside an expanded Trusted Access for Cyber (TAC) programme targeting authenticated defenders and security teams. While the initiative is framed as a defensive measure, the dual-use nature of a vulnerability-detection model introduces significant risk of adversarial inversion — where threat actors could exploit the same capabilities to discover and weaponise unpatched vulnerabilities at scale. OpenAI acknowledges this risk and states it is iteratively strengthening safeguards against jailbreaks and adversarial prompt injection as access broadens.

Anthropic Claude Mythos Sparks AI Vulnerability Storm Warning

ATLAS OWASP HIGH ▲ 7.2 Dark Reading Apr 14, 2026

The Cloud Security Alliance has issued a warning about an anticipated 'AI vulnerability storm' following the release of Anthropic's Claude Mythos model, urging CISOs to prepare defensive postures in advance of expected exploit activity. The advisory signals growing institutional concern that major LLM releases create systemic risk windows as adversaries probe new model capabilities and attack surfaces. Security leaders are being advised to treat post-release periods of frontier AI models as high-alert intervals requiring elevated monitoring and response readiness.

Jailbreaks

Stay ahead of the threat.