Researcher

149 reports

All LLM Security Agentic AI Industry News Research Supply Chain Prompt Injection First Look: Security Regulatory Jailbreaks Adversarial ML Data Poisoning Model Theft AI Security Tools Security Operations

Claude Opus 4.6 Resists 6,000 Prompt Injection Attempts

ATLAS OWASP MEDIUM ▲ 6.5 Simon Willison Jun 27, 2026

A public challenge exposing an AI email assistant to over 6,000 prompt injection attempts found that Claude Opus 4.6 successfully resisted all efforts to leak secrets or execute malicious instructions embedded in emails. While the result suggests frontier model training against injection attacks is meaningfully improving, security researchers caution that the absence of a successful attack under constrained conditions does not constitute a security guarantee. The author and Hacker News community both note that sophisticated or novel attack vectors could still break through, and irreversible-damage scenarios should not rely solely on model-level defences.

GitHub Releases Copilot Agentic Harness Evaluation

FIRST LOOK ATLAS OWASP MEDIUM ▲ 6.2 GitHub Blog Jun 26, 2026

GitHub has published an evaluation of its Copilot agentic harness, detailing how the orchestration layer performs across multiple underlying models and coding tasks — effectively documenting the architecture of an autonomous, multi-step code generation and execution system. For defenders, this transparency reveals an orchestration surface where prompt injection, supply chain manipulation, and model-switching logic can be targeted across a broader set of model backends than previously understood. Security teams should treat the harness itself as a critical trust boundary, since compromising task routing or model selection logic could silently redirect agentic workflows to less-safe or adversary-controlled model endpoints.

Google DeepMind Releases AI Agent Attack Taxonomy

FIRST LOOK ATLAS OWASP HIGH ▲ 8.7 SecurityWeek Jun 25, 2026

Google DeepMind researchers have released a structured taxonomy categorising adversarial attacks against autonomous AI agents into six classes — content injection, semantic manipulation, cognitive state poisoning, behavioural control, systemic, and human-in-the-loop traps — formalising an emerging threat model for agentic AI systems. For defenders, this framework codifies attack paths that exploit the agent's inability to distinguish trusted instructions from attacker-controlled data ingested from web pages, emails, documents, and tool outputs. NIST evaluation data cited in the research shows malicious instruction injection succeeded in 57% of tested agent hijacking scenarios on average, underscoring that these are active, high-yield attack vectors rather than theoretical concerns.

Anthropic's Mythos AI Breached Classified US Government Systems in Hours

ATLAS OWASP CRITICAL ▲ 9.1 SecurityWeek Jun 24, 2026

Anthropic's Mythos AI model identified vulnerabilities in classified US government computer systems within hours during a government-sanctioned testing exercise under Project Glasswing. A senior US official confirmed the findings to the Associated Press, corroborating statements made by Sen. Mark Warner that the model 'broke into almost all of our classified systems.' The incident marks a landmark demonstration of AI-enabled offensive cyber capability at the highest sensitivity levels of government infrastructure.

Anthropic Enhances AI Agent Skill Scanner Security

FIRST LOOK ATLAS OWASP CRITICAL ▲ 9.2 The Hacker News Jun 24, 2026

Security firm AIR demonstrated that a malicious AI agent skill, disguised as a Google Stitch landing-page builder, passed every major skill scanner including Cisco's, NVIDIA's, and skills.sh integrations, reaching approximately 26,000 agents before its payload was activated. The attack exploits a structural gap: scanners evaluate a static package at submission time, while the external URL the skill instructs the agent to fetch can be silently swapped post-install to deliver arbitrary instructions. Defenders relying on marketplace reputation signals, GitHub star counts, or one-time scanner verdicts to gatekeep agent skills have no meaningful protection against this class of supply-chain attack.

LLM Role Confusion Attack Bypasses Safety at 61%

ATLAS OWASP HIGH ▲ 8.2 Simon Willison Jun 23, 2026

New research from Ye, Cui, and Hadfield-Menell demonstrates that LLMs prioritise the stylistic format of text over its structural role tags, enabling attackers to craft injected content that mimics internal reasoning blocks and bypasses safety guardrails. The study found attack success rates of 61% when injected text stylistically matched model-internal formats, dropping to just 10% after 'destyling'. The authors conclude that without genuine role perception in models, prompt injection defences will remain fundamentally reactive.

OpenAI Launches Patch the Planet Vulnerability Initiative

FIRST LOOK ATLAS OWASP MEDIUM ▲ 5.8 TechCrunch AI Jun 23, 2026

OpenAI has partnered with Trail of Bits to launch 'Patch the Planet,' an initiative using AI-assisted tooling (including Codex Security) to help open-source maintainers find and patch vulnerabilities at scale. While the defensive intent is clear, the program introduces new attack surface considerations: AI-generated patches applied to widely-used open-source projects create a high-value supply chain target, and the triage/remediation pipeline itself could be manipulated to introduce subtle flaws. Defenders should monitor open-source dependencies that receive AI-assisted patches and assess the integrity guarantees of the remediation workflow.

AutoJack: Microsoft AutoGen Studio RCE via MCP WebSocket

ATLAS OWASP HIGH ▲ 8.5 BleepingComputer Jun 23, 2026

A three-flaw vulnerability chain dubbed AutoJack in Microsoft's AutoGen Studio allowed attackers to execute arbitrary commands on a developer's host system by manipulating a browsing AI agent into connecting to a malicious webpage. The attack exploited missing authentication on MCP WebSocket routes combined with unsanitised base64-encoded parameters to launch arbitrary processes. Microsoft confirmed the flaw was patched before any PyPI release, limiting exposure to developers building directly from the main GitHub branch.

OpenAI's ChatGPT Image Generation Fails Content Moderation

FIRST LOOK ATLAS OWASP HIGH ▲ 8.2 OpenAI (via HN) Jun 22, 2026

Mindgard researchers demonstrated that ChatGPT's image generation pipeline can be manipulated through an indirect, socially-engineered prompt to produce violent and sexually explicit content without users directly requesting it, exposing a significant failure in OpenAI's content moderation controls. Defenders and enterprise operators of ChatGPT-integrated products face a newly validated attack class where innocuous-looking prompt patterns — potentially spreading virally — can systematically strip safety guardrails from image generation. This finding signals that content filter bypasses in multimodal systems are reproducible at scale, raising urgent questions about the adequacy of output-layer filtering as a sole defence mechanism.

Bayer and Thoughtworks Ship PRINCE Agentic RAG Platform

FIRST LOOK ATLAS OWASP HIGH ▲ 7.2 HN AI Security Jun 22, 2026

Bayer AG and Thoughtworks have published a detailed case study on PRINCE, a production agentic RAG system combining multi-agent orchestration, Text-to-SQL, and human-in-the-loop workflows to answer complex pharmaceutical preclinical research questions and draft regulatory documents. The system's architecture — spanning intent clarification, planning, retrieval, reflection, and writing agents with access to decades of safety study data — introduces a broad attack surface including prompt injection across agent boundaries, SQL injection via natural language, and sensitive data exfiltration through compromised agent outputs. Defenders evaluating similar agentic platforms should treat each inter-agent handoff as a trust boundary requiring independent validation and focus on data leakage controls given the sensitivity of preclinical regulatory data.

Anthropic Launches Claude Code with Local Memory Layer

FIRST LOOK ATLAS OWASP MEDIUM ▲ 5.8 Anthropic (via HN) Jun 22, 2026

Recall is an open-source, fully-local memory layer for Anthropic's Claude Code that persists and summarises project context across coding sessions without sending data to external services. For defenders, the introduction of a persistent, file-based context store creates a new attack surface: a poisoned or tampered memory file can silently inject malicious instructions into every subsequent Claude Code session. Security teams should treat the local memory store as a trusted-input boundary and apply appropriate file-integrity and access controls.

OpenAI Ships GPT-5.5 Instant with Health Intelligence

FIRST LOOK ATLAS OWASP MEDIUM ▲ 5.8 OpenAI Blog Jun 21, 2026

OpenAI has upgraded ChatGPT's health and wellness response capabilities via GPT-5.5 Instant, incorporating stronger reasoning, physician-informed evaluations, and improved contextual understanding for medical queries. This expansion into high-stakes health guidance raises meaningful concerns for defenders, as improved fluency and authority in medical responses increases the risk of user overreliance and lowers the perceived threshold for trusting AI-generated health advice. Security and trust-safety teams should evaluate how this capability interacts with prompt injection, social engineering chains, and the broader risk of AI-mediated medical misinformation at scale.

GitHub Ships Data Analytics Agent Built on Copilot

FIRST LOOK ATLAS OWASP MEDIUM ▲ 6.8 GitHub Blog Jun 20, 2026

GitHub has published a detailed engineering account of how it built an internal data analytics agent using GitHub Copilot, exposing the architectural patterns — including natural language-to-SQL translation, autonomous tool invocation, and internal data access — that underpin such systems. For defenders, this blueprint highlights concrete risks around prompt injection into analytics pipelines, excessive agency over sensitive internal datasets, and the challenge of auditing LLM-generated queries before execution. Organisations adopting similar agentic analytics patterns should treat this as a reference threat model rather than a safe-to-copy architecture.

AutoGen Studio RCE: AutoJack Exploit Chain Targets Developers

ATLAS OWASP HIGH ▲ 9.1 The Hacker News Jun 20, 2026

Microsoft researchers disclosed AutoJack, an exploit chain targeting AutoGen Studio's MCP WebSocket endpoint that allows a single malicious web page to execute arbitrary commands on a developer's host machine via an AI browsing agent. The attack chains three distinct weaknesses — localhost trust bypass, missing authentication on MCP paths, and unsanitised command execution — requiring no credentials or user interaction beyond the agent loading the attacker's URL. While the vulnerable handler was not included in stable PyPI releases, it shipped in two pre-release builds that remain unyanked, leaving anyone who installed those versions exposed.

Delphi Ships AI Karamo Brown Clone for Kē Wellness App

FIRST LOOK ATLAS OWASP MEDIUM ▲ 6.2 TechCrunch AI Jun 19, 2026

Karamo Brown's Kē wellness app deploys an AI digital clone of the celebrity — voice, persona, and advisory content — built by Delphi from interviews, podcasts, and public clips, enabling real-time conversational coaching at scale. For defenders, celebrity-clone architectures introduce layered risks: the training corpus is largely public and manipulable, the voice synthesis surface is exploitable for deepfake derivation, and the mental-health context creates elevated harm potential if the persona is hijacked or jailbroken. Security teams evaluating similar deployments should treat the persona boundary as a primary control point, since users in vulnerable emotional states are disproportionately exposed to manipulation if guardrails fail.

Researcher

Stay ahead of the threat.