Attack Context

Black-box Research

Attacks requiring no knowledge of model internals

536 matching entries out of 736 in this category

Black-box research results

536 entries

Visual prompt injection crosses document trust boundaries

Published Sep 3, 2026Analyzed Sep 9, 2026Source: arXiv

Untrusted document images can redirect vision-language agents across instruction and tool-authorization boundaries. Repeat-After-Me evaluates six victim models on constructed document tasks.

Repeat-After-Me: Black-Box Adaptive Visual Prompt Injection

Evaluated models: claude-opus-4-7, GPT-5.5, Gemini 3.1 Pro+3 more

claude-opus-4-7
GPT-5.5
Gemini 3.1 Pro
Qwen3.6-27B
Qwen3-VL 32B Instruct
InternVL3.5 38B Instruct

Black-Box System Prompt Leakage in Real-World LLM Applications

Published Jun 17, 2026Analyzed Jul 20, 2026Source: arXiv

The paper reports a reproducible black-box evaluation showing that adversarial user queries can cause deployed LLM applications to reveal hidden system prompts. In the authors’ measurement of 1,200 applications across six commercial platforms, 1,064 applications leaked prompt content (81.0%–93.5% per anonymized platform). This is a paper-reported result, not independently verified here. LeakBench and the official artifact repository provide defensive benchmark materials for controlled testing…

Understanding and Mitigating Prompt Leaking Attacks in Real-World LLM-Based Applications

Evaluated models: Llama 2 7B Chat, Llama 3.1 8B Instruct, Mistral 7B Instruct v0.3+5 more

Llama 2 7B Chat
Llama 3.1 8B Instruct
Mistral 7B Instruct v0.3
Qwen 3 4B Instruct 2507
Qwen 3 32B
Qwen 2.5 72B Instruct
Llama 3.3 70B Instruct
Qwen 3 30B-A3B Instruct-2507

Automated indirect prompt injection against tool-calling agents

Published Jun 9, 2026Analyzed Jul 21, 2026Source: arXiv

The paper presents a concrete, reproducible security evaluation in which attacker-controlled instructions embedded in retrieved external content steer stateful, tool-calling LLM agents toward unauthorized actions. It adapts white-box GCG and black-box TAP to AgentDojo and evaluates single-task and task-universal attacks across 80 task pairs in four domains. The reported results show that semantic black-box optimization can discover functional prompt injections more effectively than…

Assessing Automated Prompt Injection Attacks in Agentic Environments

Evaluated models: Gemma3-4B Instruct, Qwen 3 4B Instruct, GPT-5+6 more

Gemma3-4B Instruct
Qwen 3 4B Instruct
GPT-5
GPT-5 Mini
Qwen 3 32B
Qwen3-235B-A22B (MoE)
Qwen3-235B-A22B-Thinking (MoE)
Claude Sonnet 4.5
Gemini 2.5 Flash

Simulated Multi-Turn Priming Bypasses LLM Safety Alignment

Published May 4, 2026Analyzed Jul 21, 2026Source: arXiv

The paper reports a reproducible black-box jailbreak evaluation in which an evolutionary search generates simulated, role-labeled multi-turn dialogue histories and submits them as a single request. Five semantic mutators and graded judge feedback optimize conversational priming that can induce harmful responses despite direct-request refusals. Results are paper-reported and not independently verified.

ContextualJailbreak: Evolutionary Red-Teaming via Simulated Conversational Priming

Evaluated models: Qwen 3 8B, GPT-oss 20B, GPT-oss 120B+7 more

Qwen 3 8B
GPT-oss 20B
GPT-oss 120B
Llama 3.1 70B
GPT-4o Mini
GPT-5
Gemini 3 Flash
claude-opus-4-7
Claude Sonnet 4.6
all-MiniLM-L6-v2

Diverse VLA Linguistic Fragility

Published Apr 7, 2026Analyzed Apr 10, 2026Source: arXiv

Vision-Language-Action (VLA) models suffer from a severe linguistic fragility vulnerability where semantically equivalent but structurally complex adversarial instructions cause catastrophic failures in visual grounding and geometric reasoning. Attackers can reliably induce physical execution failures in robotic manipulation tasks by applying semantic-preserving linguistic variations, such as synonymous rephrasing, syntactic restructuring, or the addition of fine-grained compositional…

Uncovering Linguistic Fragility in Vision-Language-Action Models via Diversity-Aware Red Teaming

Evaluated models: Pi-Zero, OpenVLA 7B, 3D-Diffuser Actor

Agent Goal Reframing Exploit

Published Apr 6, 2026Analyzed Apr 10, 2026Source: arXiv

LLM agents deployed with tool access are vulnerable to goal-reframing prompt injection, which reliably bypasses explicit rule-following and safety instructions. By injecting language that recasts a standard task as a "puzzle," "Capture The Flag (CTF)," or "easter egg" hunt, an attacker can manipulate the agent's task-frame alignment. The agent does not override security rules; instead, it reinterprets exploitative actions—such as exploring hidden directories, bypassing test runners, or reading…

Mapping the Exploitation Surface: A 10,000-Trial Taxonomy of What Makes LLM Agents Exploit Vulnerabilities

Evaluated models: Claude Sonnet 4, GPT-4.1, GPT-5 Mini+4 more

Claude Sonnet 4
GPT-4.1
GPT-5 Mini
o4-mini
DeepSeek V3
GPT-5.4-mini
GPT-5.4 Nano

Maladaptive Therapeutic Reinforcement

Published Apr 6, 2026Analyzed Apr 10, 2026Source: arXiv

Large Language Models (LLMs) aligned for helpfulness and empathy are vulnerable to a Persona-based Client Simulation Attack (PCSA) that exploits the model's inability to distinguish therapeutic empathy from maladaptive validation. By embedding harmful intents within coherent, multi-turn psychological counseling narratives and employing clinical resistance strategies (such as intellectualization or metaphorical expression), attackers can compel the model to prioritize rapport-building over…

Do No Harm: Exposing Hidden Vulnerabilities of LLMs via Persona-based Client Simulation Attack in Psychological Counseling

Evaluated models: GPT-3.5 Turbo, GPT-5.1, Llama 3.1 8B+5 more

GPT-3.5 Turbo
GPT-5.1
Llama 3.1 8B
Llama 3.1 70B
Crispers 7B
Psycho 8B
Qwen 3 14B
Qwen 2.5 72B

Indirect Agent Privilege Exposure

Published Apr 4, 2026Analyzed Apr 11, 2026Source: arXiv

Autonomous LLM agents deployed in dynamic, multi-step tool-calling environments are highly vulnerable to Indirect Prompt Injections (IPI) embedded in external content. Surface-level defensive prompts and monitoring mechanisms (such as Prompt Warning, the Sandwich Method, Spotlighting, Keyword Filtering, and LLM-as-a-Judge) consistently fail to prevent exploitation and occasionally exacerbate the vulnerability by introducing adversarial distraction. While compromised agents exhibit…

Your Agent is More Brittle Than You Think: Uncovering Indirect Injection Vulnerabilities in Agentic LLMs

Evaluated models: Qwen 2.5 14B, Qwen 2.5 32B, Qwen 3 4B+6 more

Qwen 2.5 14B
Qwen 2.5 32B
Qwen 3 4B
Qwen 3 8B
Qwen 3 14B
Llama 3 8B
GLM 4 9B
Gemma 3 12B
Mistral 7B

Multi-Strategy Prompt Evasion

Published Apr 4, 2026Analyzed Apr 10, 2026Source: arXiv

A vulnerability in LLM input filtering mechanisms allows attackers to bypass keyword, semantic, and state-of-the-art intent-aware defenses using composite prompt injections. By combining Obfuscation (OBF) techniques with Semantic/Social manipulation—specifically Emotional Manipulation (EM) or Reward Framing (RF)—attackers exploit a "representation gap" between the model and the defense. The underlying LLM decodes the obfuscated payload, while the defense mechanisms fail to parse the raw…

AttackEval: A Systematic Empirical Study of Prompt Injection Attack Effectiveness Against Large Language Models

Evaluated models: Not reported

Agent Implicit Doc Execution

Published Apr 3, 2026Analyzed Apr 10, 2026Source: arXiv

LLM-based coding agents are vulnerable to Document-Driven Implicit Payload Execution (DDIPE) via supply-chain poisoning of third-party agent skills. Attackers can embed malicious logic directly into legitimate-looking code examples and configuration templates within skill documentation files (e.g., SKILL.md). Because coding agents ingest this metadata into their context windows and treat the documentation as an authoritative reference, the underlying LLM silently reproduces and executes the…

Supply-Chain Poisoning Attacks Against LLM Coding Agent Skill Ecosystems

Evaluated models: Claude Sonnet 4.6, GLM-4.7, MiniMax M2.5+2 more

Claude Sonnet 4.6
GLM-4.7
MiniMax M2.5
GPT-5.4
Gemini 2.5 Pro

Related research categories