Architecture Components

Multimodal Research

Security research involving multimodal AI systems

176 matching entries

Multimodal research results

176 entries

Benign LLM Secondary Risks

Published Jun 14, 2025Analyzed Feb 21, 2026Source: arXiv

Large Language Models (LLMs) and Multimodal Large Language Models (MLLMs) are vulnerable to "Secondary Risks," a class of non-adversarial failures where the model generates harmful, misleading, or unsafe outputs in response to benign, non-malicious user prompts. Unlike jailbreaks which require adversarial inputs, secondary risks arise from imperfect generalization and alignment failures during standard interactions. This vulnerability manifests primarily in two primitives: 1. Excessive…

Exploring the Secondary Risks of Large Language Models

Evaluated models: GPT-4o, Claude 3.7 Sonnet, GPT-4 Turbo+9 more

GPT-4o
Claude 3.7 Sonnet
GPT-4 Turbo
Gemini 2.0 Pro
DeepSeek V3
Llama 3.3 70B
Qwen 2.5 32B
Phi-4
Gemma 2 27B IT
LLaVA OneVision Qwen2 72B
Pixtral 12B
MiniCPM-o 2.6

JailFlip Implicit Harm

Published Jun 9, 2025Analyzed Dec 8, 2025Source: arXiv

A vulnerability exists in the safety alignment mechanisms of Large Language Models (LLMs) (including GPT-4, Claude 3, Gemini, and Qwen families) leading to "Implicit Harm." Unlike traditional jailbreaks that use overtly harmful queries, this vulnerability allows remote attackers to coerce the model into providing factually incorrect, plausible, and dangerous responses to benign-looking inputs. By employing "JailFlip" techniques—specifically constructed affirmative-type or denial-type queries…

Beyond Jailbreaks: Revealing Stealthier and Broader LLM Security Risks Stemming from Alignment Failures

Evaluated models: GPT-4.1, GPT-4.1 Mini, GPT-4o+3 more

GPT-4.1
GPT-4.1 Mini
GPT-4o
GPT-4o Mini
Qwen Plus
Qwen Turbo

Phantom Token User Deception

Published May 30, 2025Analyzed Jan 14, 2026Source: arXiv

Large Language Models (LLMs) that utilize byte-stream parsing or structural extraction to process PDF files—specifically the OpenAI GPT and Anthropic Claude families—are vulnerable to adversarial text injection via imperceptible "phantom tokens." This vulnerability exploits the disconnect between how PDF viewers render documents for humans (visual layer) and how LLMs extract text from the PDF operator stream (data layer). Attackers can manipulate standard PDF text-showing operators (TJ and Tj)…

TRAPDOC: Deceiving LLM Users by Injecting Imperceptible Phantom Tokens into Documents

Evaluated models: GPT-4, o4-mini

Pretraining Modality Gap Jailbreak

Published May 30, 2025Analyzed Dec 30, 2025Source: arXiv

Large Vision-Language Models (LVLMs) that utilize a projection layer (adapter) to bridge a vision encoder and a Large Language Model (LLM) contain a vulnerability stemming from the "Modality Gap"—a distributional distance between image and text token embeddings. This gap allows the visual modality to bypass the safety alignment (RLHF/instruction tuning) of the backbone LLM. Attackers can trigger harmful, toxic, or illegal responses to queries that would be refused in text-only contexts by…

Bootstrapping LLM Robustness for VLM Safety via Reducing the Pretraining Modality Gap

Evaluated models: LLaVA 7B, Vicuna 7B

VLM Navigation Object Fusion

Published May 29, 2025Analyzed Dec 30, 2025Source: arXiv

The Vision-Language Model (VLM) perception module in Vision-and-Language Navigation (VLN) agents is vulnerable to adversarial 3D object injection via the Adversarial Object Fusion (AdvOF) framework. An attacker can generate physically plausible 3D objects with adversarial perturbations capable of deceiving the agent's VLM across multiple viewing angles and distances. The vulnerability exists due to a misalignment between 3D physical manipulations and the agent's 2D image perception, combined…

Disrupting Vision-Language Model-Driven Navigation Services via Adversarial Object Fusion

Evaluated models: Not reported

Hybrid Agent Prompt Injection

Published May 28, 2025Analyzed Dec 9, 2025Source: arXiv

Computer-Use Agents (CUAs) powered by Large Language Models (LLMs) operating in hybrid Web-OS environments are vulnerable to indirect prompt injection. Attackers can embed malicious natural language or code instructions within legitimate web content (e.g., social media forums, chat applications, shared cloud documents) that the agent processes during benign task execution. Due to the agent's inability to distinguish between trusted user instructions and untrusted environmental data, the CUA…

RedTeamCUA: Realistic Adversarial Testing of Computer-Use Agents in Hybrid Web-OS Environments

Evaluated models: Claude 3.5 Sonnet, Claude 3.7 Sonnet, GPT-4o

VLM Latent Boundary Crossing

Published May 26, 2025Analyzed Dec 30, 2025Source: arXiv

Vision-Language Models (VLMs) contain a vulnerability in their multimodal fusion layers where safety-relevant information is linearly separable in the latent space. This allows for a "JailBound" attack, which exploits the implicit internal safety decision boundary. The attack proceeds in two stages: (1) Safety Boundary Probing, where attackers approximate the internal decision hyperplane by training layer-wise logistic regression classifiers on the fusion representations of safe versus unsafe…

JailBound: Jailbreaking Internal Safety Boundaries of Vision-Language Models

Evaluated models: Llama 3.2 11B Vision Instruct, Qwen 2.5 VL 7B Instruct, MiniGPT-4+3 more

Llama 3.2 11B Vision Instruct
Qwen 2.5 VL 7B Instruct
MiniGPT-4
GPT-4o
Gemini 2.0 Flash
Claude 3.5 Sonnet

Dynamic Prompt Jailbreak

Published May 25, 2025Analyzed May 31, 2025Source: arXiv

GhostPrompt demonstrates a vulnerability in multimodal safety filters used with text-to-image generative models. The vulnerability allows attackers to bypass these filters by using a dynamic prompt optimization framework that iteratively generates adversarial prompts designed to evade both text-based and image-based safety checks while preserving the original, harmful intent of the prompt. This bypass is achieved through a combination of semantically aligned prompt rewriting and the injection…

GhostPrompt: Jailbreaking Text-to-image Generative Models based on Dynamic Optimization

Evaluated models: DALL-E 3, DeepSeek V3, Flux Schnell+5 more

DALL-E 3
DeepSeek V3
Flux Schnell
GPT-3.5 Turbo
GPT-4.1
InternVL 2 2B
Qwen 2.5 7B Instruct
ShieldLM 7B

Audio Adversarial Jailbreak

Published May 24, 2025Analyzed May 31, 2025Source: arXiv

A vulnerability in SpeechGPT allows bypassing safety filters through adversarial audio prompts crafted by a white-box token-level attack. The attacker leverages knowledge of SpeechGPT's internal speech tokenization process to generate adversarial token sequences, which are then synthesized into audio. These audio prompts elicit restricted or harmful outputs the model would normally suppress. The attack's effectiveness relies on the model's discrete audio token representation and does not…

Audio Jailbreak Attacks: Exposing Vulnerabilities in SpeechGPT in a White-Box Framework

Evaluated models: SpeechGPT

Hidden Image Jailbreak

Published May 22, 2025Analyzed May 31, 2025Source: arXiv

Multimodal large language models (MLLMs) are vulnerable to implicit jailbreak attacks that leverage least significant bit (LSB) steganography to conceal malicious instructions within images. These instructions are coupled with seemingly benign image-related text prompts, causing the MLLM to execute the hidden malicious instructions. The attack bypasses existing safety mechanisms by exploiting cross-modal reasoning capabilities.

Implicit Jailbreak Attacks via Cross-Modal Information Concealment on Vision-Language Models

Evaluated models: Gemini 1.5 Pro, Gemini 2.5 Pro, GPT-4.5+3 more

Gemini 1.5 Pro
Gemini 2.5 Pro
GPT-4.5
GPT-4o
InternVL 2 8B
Qwen 2.5 VL 72B Instruct

Related research categories