Attack Context

Black-box Research

Attacks requiring no knowledge of model internals

42 matching entries out of 736 in this category

Black-box research results

42 entries

Visual prompt injection crosses document trust boundaries

Published Sep 3, 2026Analyzed Sep 9, 2026Source: arXiv

Untrusted document images can redirect vision-language agents across instruction and tool-authorization boundaries. Repeat-After-Me evaluates six victim models on constructed document tasks.

Repeat-After-Me: Black-Box Adaptive Visual Prompt Injection

Evaluated models: claude-opus-4-7, GPT-5.5, Gemini 3.1 Pro+3 more

claude-opus-4-7
GPT-5.5
Gemini 3.1 Pro
Qwen3.6-27B
Qwen3-VL 32B Instruct
InternVL3.5 38B Instruct

Observation Poisons Agent Memory

Published Apr 3, 2026Analyzed Apr 10, 2026Source: arXiv

Memory-augmented LLM web agents utilizing raw trajectory memory are vulnerable to Environment-injected Trajectory-based Agent Memory Poisoning (eTAMP). Attackers can embed malicious instructions within user-generated web content (e.g., product pages, forum posts). When the agent processes this content during a routine task, the instructions are passively ingested into its raw trajectory memory. During subsequent, entirely separate tasks on different websites, semantic retrieval mechanisms pull…

Poison Once, Exploit Forever: Environment-Injected Memory Poisoning Attacks on Web Agents

Evaluated models: GPT-5 Mini, GPT-5.2, GPT-oss 120B+3 more

GPT-5 Mini
GPT-5.2
GPT-oss 120B
Qwen 2.5 VL 72B
Qwen3-VL-32B
Qwen3.5-122B-A10B

Invisible Visual Prompt Injection

Published Mar 31, 2026Analyzed Apr 11, 2026Source: arXiv

An imperceptible visual prompt injection vulnerability in Multimodal Large Language Models (MLLMs) allows attackers to execute precise command-hijacking via a Covert Triggered dual-Target Attack (CoTTA). By embedding a bounded, learnable textual overlay ($L_\infty$ norm bound $\varepsilon \le 16$) and adversarial noise into an input image, the attack forces the source image's internal feature representation to align with both the textual and visual embeddings of an attacker-specified…

Adversarial Prompt Injection Attack on Multimodal Large Language Models

Evaluated models: GPT-4o, GPT-5

Agentic Robot Instruction Attack

Published Mar 26, 2026Analyzed Apr 10, 2026Source: arXiv

Vision-Language-Action (VLA) models are vulnerable to targeted, low-budget textual perturbations in their natural-language instruction inputs, which can maliciously alter sequential decision-making and downstream physical robotic behavior. Because VLA policies tightly couple language, perception, and control, bounded edits—such as character-level typos, token attribute swaps, or prompt-level uncertainty clauses—propagate through the model's execution trajectory. This allows a black-box…

SABER: A Stealthy Agentic Black-Box Attack Framework for Vision-Language-Action Models

Evaluated models: Not reported

Mobile Agent Visual Spoofing

Published Feb 11, 2026Analyzed Feb 22, 2026Source: arXiv

Mobile Large Language Model (LLM) agents operating under the "Screen-as-Interface" paradigm are vulnerable to visual indirect prompt injection and state desynchronization. Agents that rely on unstructured visual data (screenshots) and Accessibility Service APIs to perceive the environment lack a mechanism to distinguish between trusted system UI elements and untrusted content (e.g., web pages, emails, or malicious overlays). An attacker can inject visual cues, fake notifications, or hidden…

Blind Gods and Broken Screens: Architecting a Secure, Intent-Centric Mobile Agent Operating System

Evaluated models: Not reported

Image Editing Visual Prompt Jailbreak

Published Feb 10, 2026Analyzed Feb 20, 2026Source: arXiv

Large Image Editing Models (LIEMs) supporting vision-prompt editing are vulnerable to Vision-Centric Jailbreak Attacks (VJA). This vulnerability arises from a modality mismatch in safety alignment: while safeguards primarily analyze textual instructions for policy violations, the underlying models are capable of interpreting and executing instructions embedded directly within the visual input (e.g., typographic text drawn on the image, arrows, symbols, or specific markings). An attacker can…

When the Prompt Becomes Visual: Vision-Centric Jailbreak Attacks for Large Image Editing Models

Evaluated models: GPT Image 1.5, Gemini 3 Pro Image, Seedream 4.5+5 more

GPT Image 1.5
Gemini 3 Pro Image
Seedream 4.5
Qwen-Image-Edit-2512
Qwen-Image-Edit-Safe
BAGEL 14B
Flux 2.0 Dev 32B
LongCat-Image-Edit

Adaptive Web Agent Prompt Injection

Published Feb 9, 2026Analyzed Feb 21, 2026Source: arXiv

Large Language Model (LLM) based web agents (such as those built using the BrowserUse scaffold) are vulnerable to Indirect Prompt Injection (IPI) attacks when autonomously navigating and processing untrusted web content. Unlike standard Cross-Site Scripting (XSS), this vulnerability occurs when the LLM orchestrator consumes the DOM or visual screenshots of a webpage containing concealed or contextually disguised adversarial instructions. The LLM interprets these embedded text strings as…

MUZZLE: Adaptive Agentic Red-Teaming of Web Agents Against Indirect Prompt Injection Attacks

Evaluated models: GPT-4.1, GPT-4o, Qwen3-VL 32B Instruct

Invisible Prompt Phishing Evasion

Published Feb 5, 2026Analyzed Mar 8, 2026Source: arXiv

Multimodal LLM-based phishing detection systems are vulnerable to indirect prompt injection via "perceptual asymmetry." Attackers can embed hidden instructions within a phishing site's HTML, CSS, URLs, or rendered images that remain imperceptible to human victims but are parsed and executed by the evaluating LLM. This vulnerability allows threat actors to manipulate the LLM's contextual understanding, forcing it to misclassify malicious sites as benign (Legitimate Pretexting), trigger safety…

Clouding the Mirror: Stealthy Prompt Injection Attacks Targeting LLM-based Phishing Detection

Evaluated models: GPT-5, Grok 4 Fast Non-Reasoning, Llama 4 Maverick+1 more

GPT-5
Grok 4 Fast Non-Reasoning
Llama 4 Maverick
Gemma 3 27B

Progressive Attention LVLM Attack

Published Feb 4, 2026Analyzed Feb 22, 2026Source: arXiv

Large Vision-Language Models (LVLMs) are vulnerable to a Stage-wise Attention-Guided Attack (SAGA) that allows for the generation of highly transferable, imperceptible adversarial examples. The vulnerability stems from a positive correlation between regional cross-modal attention scores and adversarial loss sensitivity in LVLMs. An attacker can exploit this by extracting an attention map from a surrogate open-source model (e.g., Qwen3-VL) to identify high-attention "hotspots." SAGA utilizes a…

Stage-wise Attention-Guided Region Sequencing for Adversarial Attacks on Large Vision-Language Models

Evaluated models: Gemini 2.5 Flash, Gemini 3 Pro Preview, GPT-4.1+7 more

Gemini 2.5 Flash
Gemini 3 Pro Preview
GPT-4.1
GPT-5 Mini
Grok 4 Fast
LLaVA 1.5 7B
Llama 4 Maverick
Gemma 3 4B IT
Qwen3-VL 30B-A3B Instruct
Qwen3-VL 235B-A22B Instruct

Visual Object Injection

Published Jan 24, 2026Analyzed Feb 22, 2026Source: arXiv

Large Vision-Language Models (LVLMs) are vulnerable to Physical Prompt Injection Attacks (PPIA), a query-agnostic injection technique delivered via the visual modality. The vulnerability stems from the model's "Vision-Enabled Text Recognition" capabilities and "Identity Sensitivity," where the model interprets text embedded in the physical environment (e.g., printed on signs, posters, or objects) as high-priority instructions rather than passive visual data. An attacker can embed adversarial…

Physical Prompt Injection Attacks on Large Vision-Language Models

Evaluated models: GPT-4o, GPT-4o Mini, GPT-4 Turbo+7 more

GPT-4o
GPT-4o Mini
GPT-4 Turbo
Gemini 1.5 Pro Latest
Gemini 1.5 Pro 002
Gemini 1.5 Flash Latest
Claude 3.5 Sonnet Latest
Claude 3.5 Haiku 20241022
Llama 3.2 11B Vision
Llama 3.2 90B Vision Instruct

Related research categories