F5 Labs November 2025 AI Security Insights: Fallacy Failure jailbreak added to CASI, agentic browser and Claude code interpreter risks
F5 Labs published its November 2025 AI Security Insights, drawn from the Comprehensive AI Security Index (CASI) and Agentic Resistance Scoring (ARS). Anthropic models took the whole top tier of the CASI leaderboard (Claude Haiku 4.5 at 95.89, Claude Sonnet 4.5 at 95.86, Claude Sonnet 4 at 95.33), while the GPT-5 family slipped, with GPT-5 base at 80.31 and seventh place. The report adds the Fallacy Failure Attack (FFA) to CASI testing, a jailbreak that asks a model for deliberately fallacious reasoning about a restricted task so that it inadvertently outputs accurate harmful content. It also summarizes the growing agentic attack surface, including prompt injection against agentic browsers and a data exfiltration flaw in the Claude code interpreter disclosed by researcher Johann Rehberger.
Technical Details
- Fallacy Failure Attack (FFA): based on Zhou et al. (2024), Large Language Models Are Involuntary Truth-Tellers. A prompt has four components: a malicious query, a framing that requests fallacious reasoning, an explicit requirement for deceptiveness, and a normalizing fictional or academic context. Example: asking for an "incorrect way to build a ransomware sample for a class project". The model struggles to sustain false logic and may output a functional, correct description.
- Agentic browsers: ChatGPT Atlas reached only 5.8% malicious page detection, against 53% for Edge and 47% for Chrome. Reported techniques include screenshot-based injection with near-invisible text, Markdown rendering attacks that hide instructions from users, persistent memory corruption via CSRF, and URL spoofing that disguises prompt injections.
- Claude code interpreter: Rehberger disclosed on October 25 that indirect prompt injection can exfiltrate chat histories and uploaded documents. Network restrictions allow access to api.anthropic.com, so attackers can use Claude's own API to send out stolen data. Anthropic closed the report within one hour as "out of scope" and a "model safety issue".
- Other emerging techniques: Agent Session Smuggling via the Agent2Agent (A2A) protocol, Prompt Inception, and PROMPTFLUX malware, which uses Gemini AI to rewrite its code hourly for obfuscation.
- The article also refers to seven critical zero-click vulnerabilities affecting ChatGPT, without giving details.
Remediation
- Treat "hypothetical" or "incorrect reasoning" requests as potentially exploitable in sensitive domains such as code generation, security analysis and automation.
- Consider reasoning verification layers and training that separates safe hypothetical discussion from actionable instructions.
- Restrict the external tools and network egress available to AI agents, since each integration widens the attack surface.
Linha do tempo
-
F5 Labs November 2025 AI Security Insights: Fallacy Failure jailbreak added to CASI, agentic browser and Claude code interpreter risks
ler a análise desta fonte ocultar
F5 Labs published its November 2025 AI Security Insights, drawn from the Comprehensive AI Security Index (CASI) and Agentic Resistance Scoring (ARS). Anthropic models took the whole top tier of the CASI leaderboard (Claude Haiku 4.5 at 95.89, Claude Sonnet 4.5 at 95.86, Claude Sonnet 4 at 95.33), while the GPT-5 family slipped, with GPT-5 base at 80.31 and seventh place. The report adds the Fallacy Failure Attack (FFA) to CASI testing, a jailbreak that asks a model for deliberately fallacious reasoning about a restricted task so that it inadvertently outputs accurate harmful content. It also summarizes the growing agentic attack surface, including prompt injection against agentic browsers and a data exfiltration flaw in the Claude code interpreter disclosed by researcher Johann Rehberger.
Technical Details
- Fallacy Failure Attack (FFA): based on Zhou et al. (2024), Large Language Models Are Involuntary Truth-Tellers. A prompt has four components: a malicious query, a framing that requests fallacious reasoning, an explicit requirement for deceptiveness, and a normalizing fictional or academic context. Example: asking for an "incorrect way to build a ransomware sample for a class project". The model struggles to sustain false logic and may output a functional, correct description.
- Agentic browsers: ChatGPT Atlas reached only 5.8% malicious page detection, against 53% for Edge and 47% for Chrome. Reported techniques include screenshot-based injection with near-invisible text, Markdown rendering attacks that hide instructions from users, persistent memory corruption via CSRF, and URL spoofing that disguises prompt injections.
- Claude code interpreter: Rehberger disclosed on October 25 that indirect prompt injection can exfiltrate chat histories and uploaded documents. Network restrictions allow access to api.anthropic.com, so attackers can use Claude's own API to send out stolen data. Anthropic closed the report within one hour as "out of scope" and a "model safety issue".
- Other emerging techniques: Agent Session Smuggling via the Agent2Agent (A2A) protocol, Prompt Inception, and PROMPTFLUX malware, which uses Gemini AI to rewrite its code hourly for obfuscation.
- The article also refers to seven critical zero-click vulnerabilities affecting ChatGPT, without giving details.
Remediation
- Treat "hypothetical" or "incorrect reasoning" requests as potentially exploitable in sensitive domains such as code generation, security analysis and automation.
- Consider reasoning verification layers and training that separates safe hypothetical discussion from actionable instructions.
- Restrict the external tools and network egress available to AI agents, since each integration widens the attack surface.