Architecture Components

Multimodal Research

Security research involving multimodal AI systems

176 matching entries

Multimodal research results

176 entries

Benign Audio Jailbreak

Published Aug 5, 2025Analyzed Dec 9, 2025Source: arXiv

Audio-Language Models (ALMs) including Qwen2.5-Omni (3B and 7B) and Phi-4-Multimodal are vulnerable to "WhisperInject," a two-stage adversarial audio attack that bypasses safety guardrails. The vulnerability allows an attacker to inject imperceptible perturbations into benign audio inputs (e.g., a query about the weather) that force the model to generate specific harmful content. The attack utilizes a novel optimization method, Reinforcement Learning with Projected Gradient Descent (RL-PGD)…

When Good Sounds Go Adversarial: Jailbreaking Audio-Language Models with Benign Inputs

Evaluated models: Qwen 2.5 Omni 3B, Qwen 2.5 Omni 7B, Phi-4 Multimodal+2 more

Qwen 2.5 Omni 3B
Qwen 2.5 Omni 7B
Phi-4 Multimodal
Gemma 3n E2B
Gemma 3n E4B

Simulated Ensemble Jailbreak Transfer

Published Aug 3, 2025Analyzed Dec 30, 2025Source: arXiv

A vulnerability exists in the fine-tuning lifecycle of Vision-Language Models (VLMs) derived from open-source base models, termed the "grey-box threat." Adversaries with white-box access to a public base model (e.g., Qwen2-VL) can generate universal adversarial images that successfully bypass safety guardrails in proprietary, fine-tuned downstream variants. This is achieved via the Simulated Ensemble Attack (SEA), which combines two techniques: Fine-tuning Trajectory Simulation (FTS), where…

Simulated Ensemble Attack: Transferring Jailbreaks Across Fine-tuned Vision-Language Models

Evaluated models: Qwen 2 2B

Full-Spectrum Diffusion Attack

Published Jul 31, 2025Analyzed Dec 9, 2025Source: arXiv

A targeted adversarial attack vulnerability exists in Multimodal Large Language Models (MLLMs) susceptible to Adversarial-Guided Diffusion (AGD). This technique generates adversarial images by injecting targeted semantic information into the noise component of the reverse-diffusion process within a text-to-image generative model (specifically Stable Diffusion). Unlike traditional pixel-based attacks that introduce high-frequency perturbations easily removed by low-pass filtering, AGD utilizes…

Adversarial-guided diffusion for multimodal llm attacks

Evaluated models: Vicuna 13B

Prompt-Based Jailbreak Taxonomy

Published Jul 29, 2025Analyzed Aug 16, 2025Source: arXiv

Large Language Models (LLMs) and Text-to-Image (T2I) models are vulnerable to jailbreaking through prompt-based attacks that use narrative framing, semantic substitution, and context diffusion to bypass safety moderation pipelines. These attacks do not require specialized knowledge or technical expertise. Attackers can embed harmful requests within benign narratives, frame them as fictional or professional inquiries, or use euphemistic language to circumvent input filters and output…

Anyone Can Jailbreak: Prompt-Based Attacks on LLMs and T2Is

Evaluated models: Not reported

LVLM Vision-Triggered Resource Exhaustion

Published Jul 24, 2025Analyzed Jul 28, 2025Source: arXiv

A resource consumption vulnerability exists in multiple Large Vision-Language Models (LVLMs). An attacker can craft a subtle, imperceptible adversarial perturbation and apply it to an input image. When this image is processed by an LVLM, even with a benign text prompt, it forces the model into an unbounded generation loop. The attack, named RECALLED, uses a gradient-based optimization process to create a visual perturbation that steers the model's text generation towards a predefined…

Resource Consumption Red-Teaming for Large Vision-Language Models

Evaluated models: LLaVA 1.5 7B, LLaVA 1.5 13B, Qwen 2.5 VL 3B Instruct+4 more

LLaVA 1.5 7B
LLaVA 1.5 13B
Qwen 2.5 VL 3B Instruct
Qwen 2.5 VL 7B Instruct
Qwen 2.5 VL 32B Instruct
InstructBLIP Vicuna 7B
InstructBLIP Vicuna 13B

Patching Induces Emergent Deception

Published Jul 12, 2025Analyzed Dec 30, 2025Source: arXiv

Adversarial Activation Patching enables the induction of emergent deceptive behaviors in safety-aligned transformer-based Large Language Models (LLMs). By extracting intermediate activations ($A_{d}$) generated during the processing of a deceptive or harmful prompt and injecting them into the forward pass of a benign target prompt ($x_{t}$) at specific layers (specifically mid-layers, e.g., 5-10 in 32-layer architectures), an attacker can manipulate the model's internal reasoning circuits…

Adversarial activation patching: A framework for detecting and mitigating emergent deception in safety-aligned transformers

Evaluated models: GPT-4

Adversarial ROI Privacy Attack

Published Jul 11, 2025Analyzed Dec 30, 2025Source: arXiv

Vision-Language Models (VLMs) utilizing Transformer-based visual encoders (specifically CLIP and EVA-CLIP variants) are vulnerable to a targeted adversarial attack dubbed "VIP" (Visual Information Protection). This vulnerability allows an attacker to manipulate the model's internal attention mechanism to create a "blind spot" within a specific Region of Interest (ROI) of an input image. By optimizing an additive image perturbation ($\delta$), the attack minimizes the attention weights and…

VIP: Visual Information Protection through Adversarial Attacks on Vision-Language Models

Evaluated models: InstructBLIP, Vicuna 7B

Airborne Audio LLM Hijack

Published Jul 7, 2025Analyzed Jan 14, 2026Source: arXiv

Audio-based Large Language Models (ALLMs), specifically Qwen2-Audio, are vulnerable to over-the-air adversarial audio attacks. An attacker with white-box access can generate robust adversarial audio perturbations using gradient-based optimization combined with audio augmentation techniques (specifically SpecAugment, translation, and additive noise). These perturbations, when played through a speaker in the physical environment, manipulate the ALLM processing the audio via a microphone. This…

Attacker's Noise Can Manipulate Your Audio-based LLM in the Real World

Evaluated models: Not reported

Forged Assistant Message Jailbreak

Published Jul 7, 2025Analyzed Jan 14, 2026Source: arXiv

A vulnerability termed "Trojan Horse Prompting" exists in conversational multimodal models, specifically demonstrated on Google’s Gemini-2.0-flash-preview-image-generation. The vulnerability allows an attacker to bypass safety alignment mechanisms (RLHF and SFT) by manipulating the structural protocol of the conversational API. Unlike standard jailbreaks that manipulate the user prompt, this attack exploits "Asymmetric Safety Alignment" by forging a conversational history where the role is…

Trojan Horse Prompting: Jailbreaking Conversational Multimodal Models by Forging Assistant Message

Evaluated models: Gemini 2.0 Flash Preview Image Generation

Visual Jailbreak via Context Injection

Published Jul 3, 2025Analyzed Jul 14, 2025Source: arXiv

Multimodal Large Language Models (MLLMs) are vulnerable to visual contextual attacks, where carefully crafted images and accompanying text prompts can bypass safety mechanisms and elicit harmful responses. The vulnerability stems from the MLLM's ability to integrate visual and textual context to generate outputs, allowing attackers to create realistic scenarios that subvert safety filters. Specifically, the attack leverages image-driven context injection to construct deceptive multi-turn…

Visual Contextual Attack: Jailbreaking MLLMs with Image-Driven Context Injection

Evaluated models: Gemini 2.0 Flash, GPT-4o, GPT-4o Mini+3 more

Gemini 2.0 Flash
GPT-4o
GPT-4o Mini
InternVL 2.5 78B
LLaVA 7B Chat
Qwen 2.5 VL 72B Instruct

Related research categories