Architecture Components

Vision Research

Security issues in vision-language models

131 matching entries

Vision research results

131 entries

Cross-Modal Toxic Continuation

Published 3/1/2025

Analyzed 12/9/2025

Large Vision-Language Models (VLMs) are vulnerable to a cross-modal toxic continuation attack facilitated by reinforcement learning-tuned diffusion models. This vulnerability allows an attacker to bypass safety alignment and external guardrails (such as NSFW image filters) by pairing a specific text prefix with a "semantically adversarial" image. Unlike traditional gradient-based adversarial examples that rely on pixel noise, these images are semantically coherent but optimized via Denoising…

RedDiffuser: Auditing Multimodal Safety Failures in Vision-Language Models via Reinforced Diffusion

Evaluated models: LLaVA 1.5 7B, Gemini 1.5 Flash, Llama 3.2 11B Vision Instruct

Source: arXiv

Image-Text Jailbreak

Published 3/1/2025

Analyzed 3/8/2026

Multimodal Large Language Models (MLLMs) are vulnerable to coupled cross-modal jailbreak attacks that combine continuous visual perturbations with discrete textual manipulations. Because standard alignment and single-modality defenses (such as text-only safety tuning or isolated vision-encoder adversarial training) fail to secure the cross-modal interaction, attackers can simultaneously apply gradient-based noise (e.g., PGD) to input images and adversarial suffixes (e.g., GCG) to text prompts…

E2AT: Multimodal Jailbreak Defense via Dynamic Joint Optimization for Multimodal Large Language Models

Evaluated models: LLaVA 1.5 7B, Bunny 1.0 4B, Mplug-owl2

Source: arXiv

Metaphor-Based T2I Jailbreak

Published 3/1/2025

Analyzed 4/12/2025

A vulnerability in text-to-image (T2I) models allows bypassing safety filters through the use of metaphor-based adversarial prompts. These prompts, crafted using LLMs, indirectly convey sensitive content, exploiting the model's ability to infer meaning from figurative language while circumventing explicit keyword filters and model editing strategies.

Metaphor-based Jailbreaking Attacks on Text-to-Image Models

Evaluated models: DALL-E 3, Flux, Llama 3 8B Instruct +3 more

Source: arXiv

Probabilistic Multimodal Jailbreak

Published 3/1/2025

Analyzed 3/19/2025

Multimodal Large Language Models (MLLMs) are vulnerable to Jailbreak-Probability-based Attacks (JPA). JPA leverages a Jailbreak Probability Prediction Network (JPPN) to identify and optimize adversarial perturbations in input images, maximizing the probability of eliciting harmful responses from the MLLM, even with small perturbation bounds and few iterations. The attack operates by modifying the input image's hidden states within the MLLM to increase the predicted jailbreak probability.

Utilizing Jailbreak Probability to Attack and Safeguard Multimodal LLMs

Evaluated models: DeepSeek VL 1.3B, InstructBLIP Vicuna 13B, InternLM XComposer +2 more

Source: arXiv

Adversarial VLM Jailbreak

Published 2/1/2025

Analyzed 12/9/2025

Vision-Language Models (VLMs), specifically the LLaVA-1.5 and LLaVA-1.6 series, are vulnerable to optimization-based white-box jailbreak attacks despite standard safety alignment measures like Supervised Fine-Tuning (SFT) and Direct Preference Optimization (DPO). Attackers can craft adversarial perturbations in the image space (imperceptible noise) or latent space using Projected Gradient Descent (PGD) to manipulate the model's internal representations. These perturbations maximize the…

Adversary-Aware DPO: Enhancing Safety Alignment in Vision Language Models via Adversarial Training

Evaluated models: LLaVA 1.5 7B, LLaVA 1.6 7B

Source: arXiv

Flowchart-based LVLM Jailbreak Attack

Published 2/1/2025

Analyzed 3/19/2025

FC-Attack leverages automatically generated flowcharts containing step-by-step descriptions derived or rephrased from harmful queries, combined with a benign textual prompt, to jailbreak Large Vision-Language Models (LVLMs). The vulnerability lies in the model's susceptibility to visual prompts containing harmful information within the flowcharts, thus bypassing safety alignment mechanisms.

FC-Attack: Jailbreaking Large Vision-Language Models via Auto-Generated Flowcharts

Evaluated models: Claude 3.5 Sonnet 20240620, Gemini 1.5 Flash, GPT-4o 2024-08-06 +4 more

Source: arXiv

Single Image Universal Jailbreak

Published 2/1/2025

Analyzed 12/9/2025

Multimodal Large Language Models (MLLMs) are vulnerable to a universal adversarial attack where a single, optimized image can bypass safety alignment mechanisms across diverse textual queries. By employing gradient-based optimization on the input image pixels while propagating gradients through the vision encoder and language model, an attacker can craft a visual perturbation that coerces the model into a compliant state. When this adversarial image is present in the context, the model’s…

Universal Adversarial Attack on Multimodal Aligned LLMs

Evaluated models: LLaVA 1.5 7B, Llama 3.2 11B Vision Instruct, Phi-3.5 Vision Instruct +1 more

Source: arXiv

AD Black-Box Cascading Disruption

Published 1/1/2025

Analyzed 12/9/2025

Vision Language Models (VLMs) integrated into autonomous driving (AD) systems are vulnerable to a black-box adversarial attack method termed Cascading Adversarial Disruption (CAD). The vulnerability stems from the model's susceptibility to optimized visual perturbations that disrupt the decision-making reasoning chain (perception, prediction, and planning). Attackers can generate adversarial images or physical patches by aligning visual noise with deceptive textual semantics in the model's…

Black-box adversarial attack on vision language models for autonomous driving

Evaluated models: GPT-4, GPT-4o, InstructBLIP

Source: arXiv

LLM Risk Amplification

Published 1/1/2025

Analyzed 12/9/2025

Vision Language Models (VLMs) are vulnerable to visual prompt injection attacks via text-to-image obfuscation. While these models often possess safety guardrails for standard text-based inputs, they fail to apply equivalent safety alignment to textual instructions embedded visually within an image. An attacker can overlay malicious instructions (e.g., requests for illegal acts, hate speech) onto an image file and submit it to the model. The model’s Optical Character Recognition (OCR) or visual…

Lessons from red teaming 100 generative ai products

Evaluated models: GPT-4, Phi-3

Source: arXiv

Bimodal Black-Box Jailbreak

Published 12/1/2024

Analyzed 12/29/2024

A bimodal adversarial attack, PBI-Attack, can manipulate Large Vision-Language Models (LVLMs) into generating toxic or harmful content by iteratively optimizing both textual and visual inputs in a black-box setting. The attack leverages a surrogate LVLM to inject malicious features from a harmful corpus into a benign image, then iteratively refines both image and text perturbations to maximize the toxicity of the model’s output as measured by a toxicity detection model (Perspective API or…

BAMBA: A Bimodal Adversarial Multi-Round Black-Box Jailbreak Attacker for LVLMs

Evaluated models: GPT-4, InstructBLIP, MiniGPT-4 +1 more

Source: arXiv

Related research categories