Language Model Security Database

M3Att demonstrates a reproducible knowledge-poisoning issue in medical multimodal RAG: an attacker with limited corpus-distribution knowledge can insert paired image-text entries whose visually perturbed images are broadly retrieved and whose clinically plausible misinformation steers downstream generation. The paper evaluates both white-box and black-box retrieval optimization and reports degraded diagnostic and report-generation utility across multiple datasets, retrievers, and LVLMs. These…

LLaVA-v1.5-7B, when deployed as a vision-language autonomous agent, is highly vulnerable to adversarial image perturbations. An attacker can inject imperceptibly modified images into a web environment (such as an e-commerce storefront). When the VLM agent captures a screenshot containing the perturbed image, the visual noise forces the model to misclassify the scene and output incorrect, structured JSON actions. This allows an attacker to hijack the agent's task execution, bypassing the user's…

A vulnerability in Vision-Language Models (VLMs) relying on shared visual-textual representation spaces allows attackers to induce transferable cross-task semantic failures using an X-shaped Sparse Pixel Attack (XSPA). Attackers craft imperceptible adversarial perturbations restricted to a fixed geometric prior—two intersecting diagonal lines comprising approximately 1.76% of the image pixels. By jointly optimizing a classification objective with cross-task semantic guidance (target-semantic…

Discrete image tokenizers are vulnerable to unsupervised embedding-space adversarial attacks. Attackers can apply $\ell_p$-bounded perturbations to an input image to maximize the $\ell_2$ distance of the pre-quantization continuous embeddings produced by the tokenizer's vision encoder. This forces the vector quantizer to cross discrete cell boundaries and assign incorrect codebook vectors, fundamentally altering the resulting token sequence. Because the attack targets the pre-quantization…

Contrastive Language-Image Pre-training (CLIP) models are vulnerable to semantic-ensemble adversarial attacks. Current adversarial fine-tuning defenses for CLIP rely on minimizing the cosine similarity between an image and a single hand-crafted template (e.g., "A photo of a {label}"). This creates a vulnerability where adversarial examples (AEs) overfit to specific phrasings rather than the core class semantics. Attackers can bypass these defenses by generating semantic-aware adversarial…

Vision Language Models (VLMs) utilizing independent vision encoders (e.g., ViT) and Large Language Model (LLM) decoders are vulnerable to Split-Image Visual Jailbreak Attacks (SIVA). The vulnerability arises from an architectural and alignment discrepancy: while the vision encoder processes image fragments (splits) in isolation via constrained attention or block-diagonal masks, the LLM decoder aggregates these features via cross-attention to reconstruct the semantic content. Current safety…

Point-based 3D Vision-Language Models (VLMs), specifically PointLLM and GPT4Point, are vulnerable to white-box, gradient-based adversarial attacks. The vulnerability exists in the model's processing of 3D point cloud data, where an attacker can optimize imperceptible geometric perturbations ($\delta$) on the input point cloud ($x$) to manipulate the model's textual output. The paper identifies two specific attack vectors: 1. Vision Attack: Directly perturbs the high-dimensional visual token…

A vulnerability exists in trimodal audio-video-language models where an attacker can systematically degrade multimodal reasoning through untargeted, audio-only adversarial perturbations. By optimizing a shared perturbation $\delta$ applied to the audio channel, an attacker can manipulate internal representations—specifically targeting audio encoder embeddings and cross-modal attention mechanisms—without modifying visual or textual inputs. The attack exploits the model's reliance on the audio…

A vulnerability exists in Large Vision-Language Models (LVLMs) utilizing visual token compression mechanisms (e.g., VisionZip, VisPruner) to reduce inference latency. The vulnerability stems from an optimization-inference mismatch where standard adversarial defenses assume full-token processing, while the deployed model utilizes a subset of tokens selected via importance metrics (typically attention scores).

Large Language and Vision Assistant (LLaVA) v1.5-13B and Meta Llama 3.2 11B Vision are vulnerable to adversarial evasion attacks targeting the visual input modality. An attacker with white-box access (knowledge of model architecture and gradients) can employ Projected Gradient Descent (PGD) to generate adversarial perturbations constrained by an L-infinity norm. By maximizing the model's internal loss function with respect to the input image, the attacker can force the Vision-Language Model…