Architecture Components

Multimodal Research

Security research involving multimodal AI systems

176 matching entries

Multimodal research results

176 entries

Multimodal Risk Diffusion Jailbreak

Published Dec 8, 2024Analyzed Dec 29, 2024Source: arXiv

Multimodal Large Language Models (MLLMs) are vulnerable to a heuristic-induced multimodal risk distribution jailbreak attack. The attack successfully circumvents safety mechanisms by distributing malicious prompts across text and image modalities, preventing detection of harmful intent within either modality alone. An auxiliary LLM generates prompts to guide the target MLLM into reconstructing the malicious prompt and producing the desired harmful output.

Heuristic-Induced Multimodal Risk Distribution Jailbreak Attack for Multimodal Large Language Models

Evaluated models: Deepseek-vl7B-chat, Gemini 1.5 Pro, Glm-4v-9B+7 more

Deepseek-vl7B-chat
Gemini 1.5 Pro
Glm-4v-9B
GPT-4o-0513
Llava v1.5-7B
Llava v1.6-mistral-7B-hf
MiniGPT-4
Qwen VL Chat
Qwen VL Max
Yi-vl-34B

Multi-Modal VLM Jailbreak

Published Nov 30, 2024Analyzed Jan 26, 2025Source: arXiv

A novel jailbreak attack, Multi-Modal Linkage (MML), exploits the vulnerability in Large Vision-Language Models (VLMs) by leveraging an "encryption-decryption" scheme across text and image modalities. MML encrypts malicious queries within images (e.g., using word replacement, image transformations) to bypass initial safety mechanisms. A subsequent text prompt guides the VLM to "decrypt" the content, eliciting harmful outputs. "Evil alignment," framing the attack within a video game scenario…

Jailbreak Large Visual Language Models Through Multi-Modal Linkage

Evaluated models: Claude 3.5 Sonnet, GPT-4o, GPT-4o Mini+1 more

Claude 3.5 Sonnet
GPT-4o
GPT-4o Mini
Qwen VL Max

Visual Jailbreak via Multi-Loss

Published Nov 27, 2024Analyzed Dec 29, 2024Source: arXiv

Vision-Language Models (VLMs) are vulnerable to jailbreak attacks using carefully crafted adversarial images. Attackers can bypass safety mechanisms by generating images semantically aligned with harmful prompts, exploiting the fact that minimal cross-entropy loss during adversarial image optimization does not guarantee optimal attack effectiveness. The attack uses a multi-image collaborative approach, selecting images within a specific loss range to enhance the likelihood of successful…

Exploring Visual Vulnerabilities via Multi-Loss Adversarial Search for Jailbreaking Vision-Language Models

Evaluated models: LLaVA 2, MiniGPT-4

Image-Based Safety Snowballing

Published Nov 18, 2024Analyzed Dec 29, 2024Source: arXiv

A vulnerability exists in several Large Vision-Language Models (LVLMs) where seemingly safe images, when combined with additional safe images and prompts using a specific attack methodology (Safety Snowball Agent), can trigger the generation of unsafe and harmful content. The vulnerability exploits the models' universal reasoning abilities and a "safety snowball effect," where an initial unsafe response leads to progressively more harmful outputs.

Safe+ Safe= Unsafe? Exploring How Safe Images Can Be Exploited to Jailbreak Large Vision-Language Models

Evaluated models: GPT-4o, InternVL 2 40B, Qwen VL 2 72B+1 more

GPT-4o
InternVL 2 40B
Qwen VL 2 72B
VILA 1.5 40B

Zeroth-Order MLLM Jailbreak

Published Nov 12, 2024Analyzed Dec 29, 2024Source: arXiv

A vulnerability in multi-modal large language models (MLLMs) allows attackers to bypass safety mechanisms and elicit harmful responses using a memory-efficient zeroth-order optimization technique. The attack, termed Zer0-Jack, leverages simultaneous perturbation stochastic approximation (SPSA) with patch coordinate descent to generate malicious image inputs, even without access to the model's internal parameters (black-box setting).

Zer0-Jack: A Memory-efficient Gradient-based Jailbreaking Method for Black-box Multi-modal Large Language Models

Evaluated models: GPT-4o, Inf-mllm1, LLaVA 1.5+1 more

GPT-4o
Inf-mllm1
LLaVA 1.5
MiniGPT-4

VLM RedTeaming Jailbreak

Published Oct 29, 2024Analyzed Dec 29, 2024Source: arXiv

Large Vision-Language Models (VLMs) are vulnerable to a novel black-box jailbreak attack, IDEATOR, which leverages a separate VLM to generate malicious image-text pairs. The attacker VLM iteratively refines its prompts based on the target VLM's responses, bypassing safety mechanisms by generating contextually relevant and visually subtle malicious prompts.

IDEATOR: Jailbreaking and Benchmarking Large Vision-Language Models Using Themselves

Evaluated models: MiniGPT-4 Vicuna 13B, InstructBLIP, Chameleon+10 more

MiniGPT-4 Vicuna 13B
InstructBLIP
Chameleon
LLaVA-OneVision
MiniGPT-v2
Llama 3.2 11B Vision
Llama 3.2 90B Vision
GPT-4o Mini
GPT-4o
Gemini 1.5 Pro
Gemini 2.0 Flash
Gemini 2.0 Flash Thinking
Claude 3.5 Sonnet

Gradient Image Jailbreaks

Published Oct 4, 2024Analyzed Dec 29, 2024Source: arXiv

Multimodal fusion models, such as Chameleon models, utilize non-differentiable tokenization functions for image inputs, hindering direct gradient-based attacks. This vulnerability allows attackers with white-box access to bypass safety mechanisms by using a "tokenizer shortcut," a differentiable approximation of the tokenization process, to perform continuous optimization of image inputs. This enables the generation of adversarial images that elicit harmful responses from the model, even for…

Gradient-based jailbreak images for multimodal fusion models

Evaluated models: Chameleon 30B, Chameleon 7B, LLaVA 1.6 Llama 3

Iterative Image Jailbreak

Published Oct 4, 2024Analyzed Dec 29, 2024Source: arXiv

A Chain-of-Jailbreak (CoJ) attack allows bypassing safety mechanisms in image generation models by iteratively editing images based on a sequence of sub-queries. The attack decomposes a malicious query into multiple, seemingly benign sub-queries, each causing the model to generate and modify an image, ultimately producing harmful content. Successful attacks leverage various editing operations (insert, delete, change) on different elements (words, characters, images).

Chain-of-Jailbreak Attack for Image Generation Models via Editing Step by Step

Evaluated models: Gemini 1.5 Pro, GPT-4o, GPT-4V

Multi-Agent T2I Jailbreak

Published Aug 1, 2024Analyzed Dec 28, 2024Source: arXiv

A vulnerability allows bypassing safety filters in text-to-image (T2I) models using a multi-agent framework ("Atlas") powered by Large Language Models (LLMs). Atlas iteratively generates and refines prompts, leveraging a Vision-Language Model (VLM) to assess filter activation and an LLM to select effective prompts that maintain semantic similarity to the original, malicious prompt while evading the filter. This enables the generation of images containing unsafe content.

Jailbreaking text-to-image models with llm-based agents

Evaluated models: DALL-E 3, LLaVA 1.5 13B, Sharegpt4v-13B+4 more

DALL-E 3
LLaVA 1.5 13B
Sharegpt4v-13B
Stable Diffusion 3 Medium
Stable Diffusion v1.4
Stable Diffusion Xl Refiner
Vicuna-1.5-13B

AutoJailbreak of GPT-4V

Published Jul 23, 2024Analyzed Dec 29, 2024Source: arXiv

A vulnerability in GPT-4V's facial recognition safety mechanisms allows for automated jailbreaking attacks using Large Language Models (LLMs) to bypass safety features and elicit unintended facial identification responses. The attack, termed "AutoJailbreak," optimizes prompts through iterative refinement with an LLM "red-teaming" model, significantly increasing the attack success rate. This vulnerability exploits weaknesses in GPT-4V's prompt processing and safety alignment, allowing malicious…

Can Large Language Models Automatically Jailbreak GPT-4V?

Evaluated models: GPT-3.5 Turbo, GPT-4, GPT-4V

Related research categories