Attack Context

White-box Research

Attacks leveraging model implementation details

21 matching entries out of 228 in this category

White-box research results

21 entries

Query-Agnostic Poisoning of Medical Multimodal RAG

Published May 11, 2026Analyzed Jul 20, 2026Source: arXiv

M3Att demonstrates a reproducible knowledge-poisoning issue in medical multimodal RAG: an attacker with limited corpus-distribution knowledge can insert paired image-text entries whose visually perturbed images are broadly retrieved and whose clinically plausible misinformation steers downstream generation. The paper evaluates both white-box and black-box retrieval optimization and reports degraded diagnostic and report-generation utility across multiple datasets, retrievers, and LVLMs. These…

Knowledge Poisoning Attacks on Medical Multi-Modal Retrieval-Augmented Generation

Evaluated models: GPT-4o, GPT-5 Chat, Gemini 2.5 Flash+5 more

GPT-4o
GPT-5 Chat
Gemini 2.5 Flash
Claude Haiku 4.5
LLaVA Med
CLIP ViT-Large-Patch14-336
BGE-VL-base
SigLIP-SO400M-Patch14-384

VLM Image-Shift Jailbreak

Published Mar 18, 2026Analyzed Apr 10, 2026Source: arXiv

The integration of the visual modality in Large Vision-Language Models (VLMs) introduces a vulnerability where appending an image to a harmful text prompt induces a "jailbreak-related representation shift" in the model's internal high-dimensional space. This shift forcibly steers the model's last-token hidden state away from a designated refusal state and into a distinct jailbreak state. The vulnerability occurs because the visual modality overrides the safety alignment of the underlying…

Understanding and Defending VLM Jailbreaks via Jailbreak-Related Representation Shift

Evaluated models: LLaVA 1.5 7B, ShareGPT4V 7B, InternVL-Chat 19B

Discrete Token Subversion

Published Feb 20, 2026Analyzed Mar 9, 2026Source: arXiv

Discrete image tokenizers are vulnerable to unsupervised embedding-space adversarial attacks. Attackers can apply $\ell_p$-bounded perturbations to an input image to maximize the $\ell_2$ distance of the pre-quantization continuous embeddings produced by the tokenizer's vision encoder. This forces the vector quantizer to cross discrete cell boundaries and assign incorrect codebook vectors, fundamentally altering the resulting token sequence. Because the attack targets the pre-quantization…

On the Adversarial Robustness of Discrete Image Tokenizers

Evaluated models: Llama 2 7B

Visual Memory Multi-Turn Manipulation

Published Feb 17, 2026Analyzed Mar 8, 2026Source: arXiv

Large Vision-Language Models (LVLMs) are vulnerable to Visual Memory Injection (VMI), a stealthy targeted attack targeting multi-turn conversations. An attacker can embed an imperceptible adversarial perturbation ($L_\infty \le 8/255$) into a seemingly benign image. Because the visual input persists in the model's context throughout a multi-turn dialogue, the injected payload remains dormant. By utilizing "benign anchoring" and "context-cycling" during optimization, the attacker ensures the…

Visual Memory Injection Attacks for Multi-Turn Conversations

Evaluated models: Qwen 2.5 VL 7B Instruct, Qwen3-VL 8B Instruct, LLaVA-OneVision 1.5 8B Instruct+2 more

Qwen 2.5 VL 7B Instruct
Qwen3-VL 8B Instruct
LLaVA-OneVision 1.5 8B Instruct
Qwen-SEA-LION v4 8B VL
QoQ-Med3-VL 8B

VLM Split-Image Blindspot

Published Feb 8, 2026Analyzed Feb 22, 2026Source: arXiv

Vision Language Models (VLMs) utilizing independent vision encoders (e.g., ViT) and Large Language Model (LLM) decoders are vulnerable to Split-Image Visual Jailbreak Attacks (SIVA). The vulnerability arises from an architectural and alignment discrepancy: while the vision encoder processes image fragments (splits) in isolation via constrained attention or block-diagonal masks, the LLM decoder aggregates these features via cross-attention to reconstruct the semantic content. Current safety…

Robustness of Vision Language Models Against Split-Image Harmful Input Attacks

Evaluated models: Llama 3.2 11B

Visual Compression Mismatch Attack

Published Jan 29, 2026Analyzed Feb 22, 2026Source: arXiv

A vulnerability exists in Large Vision-Language Models (LVLMs) utilizing visual token compression mechanisms (e.g., VisionZip, VisPruner) to reduce inference latency. The vulnerability stems from an optimization-inference mismatch where standard adversarial defenses assume full-token processing, while the deployed model utilizes a subset of tokens selected via importance metrics (typically attention scores).

On the Adversarial Robustness of Large Vision-Language Models under Visual Token Compression

Evaluated models: LLaVA 1.5 7B

Point Cloud Adversarial Attack

Published Jan 10, 2026Analyzed Feb 22, 2026Source: arXiv

Point-based 3D Vision-Language Models (VLMs), specifically PointLLM and GPT4Point, are vulnerable to white-box, gradient-based adversarial attacks. The vulnerability exists in the model's processing of 3D point cloud data, where an attacker can optimize imperceptible geometric perturbations ($\delta$) on the input point cloud ($x$) to manipulate the model's textual output. The paper identifies two specific attack vectors: 1. Vision Attack: Directly perturbs the high-dimensional visual token…

On the Adversarial Robustness of 3D Large Vision-Language Models

Evaluated models: Vicuna 7B

Embodied Cross-Modal Misalignment

Published Nov 20, 2025Analyzed Jan 14, 2026Source: arXiv

OpenVLA, a Vision-Language-Action (VLA) model, contains a vulnerability regarding multimodal adversarial robustness. The model lacks sufficient cross-modal alignment stability, allowing attackers to disrupt the grounding between visual perception and linguistic instructions. By utilizing the "VLA-Fool" framework, adversaries can inject perturbations via three vectors: (1) Semantically Greedy Coordinate Gradient (SGCG), which alters specific linguistic tokens (referential cues, attributes…

When alignment fails: Multimodal adversarial attacks on vision-language-action models

Evaluated models: Not reported

VLM Imperceptible Image Jailbreak

Published Sep 24, 2025Analyzed Dec 8, 2025Source: arXiv

A vulnerability exists in Vision-Language Models (VLMs) that allows for the bypass of safety alignment mechanisms through loss-guided adversarial image perturbations. This attack, known as JaiLIP, operates entirely in the image space, requiring no textual prompt manipulation. The vulnerability is exploited by optimizing an adversarial image using a joint objective function that minimizes the Mean Squared Error (MSE) between the clean and perturbed image while maximizing the model's loss for…

JaiLIP: Jailbreaking Vision-Language Models via Loss Guided Image Perturbation

Evaluated models: GPT-4, InstructBLIP, Vicuna 13B

JailFlip Implicit Harm

Published Jun 9, 2025Analyzed Dec 8, 2025Source: arXiv

A vulnerability exists in the safety alignment mechanisms of Large Language Models (LLMs) (including GPT-4, Claude 3, Gemini, and Qwen families) leading to "Implicit Harm." Unlike traditional jailbreaks that use overtly harmful queries, this vulnerability allows remote attackers to coerce the model into providing factually incorrect, plausible, and dangerous responses to benign-looking inputs. By employing "JailFlip" techniques—specifically constructed affirmative-type or denial-type queries…

Beyond Jailbreaks: Revealing Stealthier and Broader LLM Security Risks Stemming from Alignment Failures

Evaluated models: GPT-4.1, GPT-4.1 Mini, GPT-4o+3 more

GPT-4.1
GPT-4.1 Mini
GPT-4o
GPT-4o Mini
Qwen Plus
Qwen Turbo

Related research categories