Attack Context

Black-box Research

Attacks requiring no knowledge of model internals

736 matching entries

Black-box research results

736 entries

Personalized Disinformation Jailbreak Escalation

Published Oct 14, 2025Analyzed Nov 11, 2025Source: arXiv

Appending simple demographic persona details to prompts requesting policy-violating content can bypass the safety mechanisms of Large Language Models. This technique, referred to as persona-targeted prompting, adds details such as country, generation, and political orientation to a request for a harmful narrative (e.g., disinformation). This systematically increases the jailbreak rate across most tested models and languages, in some cases by over 10 percentage points, enabling the generation…

A Multilingual, Large-Scale Study of the Interplay between LLM Safeguards, Personalisation, and Disinformation

Evaluated models: Claude 3.5 Sonnet, Gemma 2 9B IT, GPT-4o+5 more

Claude 3.5 Sonnet
Gemma 2 9B IT
GPT-4o
Grok 2
Llama 3 8B Instruct
Mistral Nemo Instruct
Qwen 2.5 7B Instruct
Vicuna 7B v1.5

Special Token Jailbreak

Published Oct 11, 2025Analyzed Oct 31, 2025Source: arXiv

Large Language Models (LLMs) that use special tokens to define conversational structure (e.g., via chat templates) are vulnerable to a jailbreak attack named MetaBreak. An attacker can inject these special tokens, or regular tokens with high semantic similarity in the embedding space, into a user prompt. This manipulation allows the attacker to bypass the model's internal safety alignment and external content moderation systems. The attack leverages four primitives: 1. Response Injection…

MetaBreak: Jailbreaking Online LLM Services via Special Token Manipulation

Evaluated models: Claude Opus 4, Gemma 2 27B IT, GPT-4.1+9 more

Claude Opus 4
Gemma 2 27B IT
GPT-4.1
Llama 3.1 405B
Llama 3.1 8B
Llama 3.3 70B Instruct
Llama Guard
Llama Guard 3 8B
Phi-4
Prompt Guard
Qwen 2.5 72B Instruct
ShieldGemma 2 27B

Pattern Enhanced Multi-Turn Jailbreaking

Published Oct 9, 2025Analyzed Nov 1, 2025Source: arXiv

Pattern Enhanced Chain of Attack (PE-CoA) shows that safety controls can be bypassed gradually across a conversation. The paper evaluates five recurring conversational patterns and finds that resistance to one pattern does not reliably generalize to others, creating a black-box, multi-turn jailbreak risk even when individual turns appear benign.

Pattern Enhanced Multi-Turn Jailbreaking: Exploiting Structural Vulnerabilities in Large Language Models

Evaluated models: Claude 3 Haiku, DeepSeek Chat, Gemini 1.5 Flash+9 more

Claude 3 Haiku
DeepSeek Chat
Gemini 1.5 Flash
Gemini 1.5 Pro
Gemini 2.0 Flash
GPT-3.5 Turbo
GPT-4o Mini
Llama 2 13B
Llama 2 7B
Llama 3 8B
Mistral 7B Instruct v0.3
Vicuna 13B v1.5

Touch-Guided Mobile Agent Jailbreak

Published Oct 9, 2025Analyzed Dec 9, 2025Source: arXiv

Large Vision-Language Model (LVLM) driven mobile agents, such as Mobile-Agent-E, are vulnerable to a touch-guided visual prompt injection attack. This vulnerability allows an attacker to hijack the agent's execution flow via a malicious Android application interface without requiring system-level privileges. The attack leverages "Non-privileged Perception Compromise," where a visual payload is embedded in the application UI and conditionally rendered only during agent-specific interaction…

Practical and Stealthy Touch-Guided Jailbreak Attacks on Deployed Mobile Vision-Language Agents

Evaluated models: GPT-4o, Gemini 2.0 Pro Exp 0205, Claude 3.5 Sonnet+3 more

GPT-4o
Gemini 2.0 Pro Exp 0205
Claude 3.5 Sonnet
Qwen VL Max
DeepSeek-VL2
LLaVA-OneVision

Underestimated LLM Security Flaws

Published Oct 7, 2025Analyzed Dec 30, 2025Source: arXiv

Mamba-2 and hybrid Transformer-Mamba-2 distilled Large Language Model (LLM) architectures exhibit a distinct architectural susceptibility to Latent Injection and ANSI Escape sequence prompt injection attacks. Comparative analysis reveals that models incorporating Mamba state-space components (specifically distilled variants like Llamba-3B and base Mamba models) fail to maintain adversarial robustness levels comparable to pure Transformer baselines (such as Llama-3.2) when subjected to indirect…

Towards reliable and practical LLM security evaluations via Bayesian modelling

Evaluated models: Llama 3.2 3B, Falcon 7B

RL-Hammer Autonomous Jailbreak

Published Oct 6, 2025Analyzed Dec 9, 2025Source: arXiv

A vulnerability exists in Large Language Model (LLM) agentic systems where automated reinforcement learning (RL) techniques can bypass advanced prompt injection defenses, including Instruction Hierarchy and SecAlign. The specific attack methodology, dubbed "RL-Hammer," utilizes Group Relative Policy Optimization (GRPO) to train an attacker model from scratch without warm-up data. The vulnerability exploits the reward sparsity in robust models by employing a "bag of tricks": removing KL…

RL Is a Hammer and LLMs Are Nails: A Simple Reinforcement Learning Recipe for Strong Prompt Injection

Evaluated models: Llama 3.1 8B Instruct, Meta-SecAlign 8B, Meta-SecAlign 70B+7 more

Llama 3.1 8B Instruct
Meta-SecAlign 8B
Meta-SecAlign 70B
GPT-4o Mini
GPT-4o
GPT-5 Mini
GPT-5
Gemini 2.5 Flash
Claude 3.5 Sonnet
Claude Sonnet 4

Adaptive Typographic Image Injection

Published Oct 5, 2025Analyzed Dec 30, 2025Source: arXiv

Multimodal agents built on Large Vision-Language Models (LVLMs) are vulnerable to adaptive typographic prompt injection attacks (AgentTypo). This vulnerability allows an attacker to execute indirect prompt injection by embedding adversarial text prompts directly into images (e.g., webpage screenshots, product photos) processed by the agent. Unlike standard visual adversarial attacks that rely on noise perturbation, this method utilizes the AgentTypo framework to perform black-box Bayesian…

AgentTypo: Adaptive Typographic Prompt Injection Attacks against Black-box Multimodal Agents

Evaluated models: GPT-4o, GPT-4V, GPT-4o Mini+2 more

GPT-4o
GPT-4V
GPT-4o Mini
Gemini 1.5 Pro
Claude 3 Opus

Overfitting-induced Benign Jailbreak

Published Oct 3, 2025Analyzed Oct 13, 2025Source: arXiv

A vulnerability exists in Large Language Models (LLMs) that support fine-tuning, allowing an attacker to bypass safety alignments using a small, benign dataset. The attack, "Attack via Overfitting," is a two-stage process. In Stage 1, the model is fine-tuned on a small set of benign questions (e.g., 10) paired with identical, repetitive refusal answers. This induces an overfitted state where the model learns to refuse all prompts, creating a sharp minimum in the loss landscape and making it…

Attack via Overfitting: 10-shot Benign Fine-tuning to Jailbreak LLMs

Evaluated models: DeepSeek R1 Distill Llama 8B, GPT-3.5 Turbo, GPT-4.1+7 more

DeepSeek R1 Distill Llama 8B
GPT-3.5 Turbo
GPT-4.1
GPT-4.1 Mini
GPT-4o
GPT-4o Mini
Llama 2 7B Chat
Llama 3 8B Instruct
Qwen 2.5 7B Instruct
Qwen 3 8B

Controlled-Release Guard Bypass

Published Oct 2, 2025Analyzed Dec 30, 2025Source: arXiv

A vulnerability termed "Controlled-Release Prompting" allows attackers to bypass lightweight input filters (prompt guards) deployed in front of Large Language Models (LLMs). The attack exploits the computational resource asymmetry between the resource-constrained guard model and the highly capable target model. Attackers encode malicious instructions using obfuscation techniques—such as substitution ciphers (Timed-Release) or verbose character descriptions (Spaced-Release)—that require…

Bypassing Prompt Guards in Production with Controlled-Release Prompting

Evaluated models: Gemini 2.5 Flash, Gemini 2.5 Pro, DeepSeek R1+2 more

Gemini 2.5 Flash
Gemini 2.5 Pro
DeepSeek R1
Grok 3
GPT-5 Mini

LLM Self-Targeted Jailbreak

Published Oct 2, 2025Analyzed Dec 8, 2025Source: arXiv

A security vulnerability exists in the safety alignment mechanisms of Large Language Models (LLMs), specifically susceptible to the "Dynamic Target Attack" (DTA). Unlike traditional gradient-based jailbreaks (e.g., GCG) that optimize adversarial suffixes toward a fixed, low-probability static target (e.g., "Sure, here is..."), DTA exploits the model's own output distribution. The attack iteratively samples candidate responses from the target model using relaxed decoding parameters (high…

Dynamic Target Attack

Evaluated models: Llama 3 8B, Llama 3.2 1B, Mistral 7B+3 more

Llama 3 8B
Llama 3.2 1B
Mistral 7B
Qwen 2.5 7B
Gemma 7B
Vicuna 7B

Related research categories