Architecture Components

Vision Research

Security issues in vision-language models

131 matching entries

Vision research results

131 entries

VLM Latent Boundary Crossing

Published May 26, 2025Analyzed Dec 30, 2025Source: arXiv

Vision-Language Models (VLMs) contain a vulnerability in their multimodal fusion layers where safety-relevant information is linearly separable in the latent space. This allows for a "JailBound" attack, which exploits the implicit internal safety decision boundary. The attack proceeds in two stages: (1) Safety Boundary Probing, where attackers approximate the internal decision hyperplane by training layer-wise logistic regression classifiers on the fusion representations of safe versus unsafe…

JailBound: Jailbreaking Internal Safety Boundaries of Vision-Language Models

Evaluated models: Llama 3.2 11B Vision Instruct, Qwen 2.5 VL 7B Instruct, MiniGPT-4+3 more

Llama 3.2 11B Vision Instruct
Qwen 2.5 VL 7B Instruct
MiniGPT-4
GPT-4o
Gemini 2.0 Flash
Claude 3.5 Sonnet

Dynamic Prompt Jailbreak

Published May 25, 2025Analyzed May 31, 2025Source: arXiv

GhostPrompt demonstrates a vulnerability in multimodal safety filters used with text-to-image generative models. The vulnerability allows attackers to bypass these filters by using a dynamic prompt optimization framework that iteratively generates adversarial prompts designed to evade both text-based and image-based safety checks while preserving the original, harmful intent of the prompt. This bypass is achieved through a combination of semantically aligned prompt rewriting and the injection…

GhostPrompt: Jailbreaking Text-to-image Generative Models based on Dynamic Optimization

Evaluated models: DALL-E 3, DeepSeek V3, Flux Schnell+5 more

DALL-E 3
DeepSeek V3
Flux Schnell
GPT-3.5 Turbo
GPT-4.1
InternVL 2 2B
Qwen 2.5 7B Instruct
ShieldLM 7B

Hidden Image Jailbreak

Published May 22, 2025Analyzed May 31, 2025Source: arXiv

Multimodal large language models (MLLMs) are vulnerable to implicit jailbreak attacks that leverage least significant bit (LSB) steganography to conceal malicious instructions within images. These instructions are coupled with seemingly benign image-related text prompts, causing the MLLM to execute the hidden malicious instructions. The attack bypasses existing safety mechanisms by exploiting cross-modal reasoning capabilities.

Implicit Jailbreak Attacks via Cross-Modal Information Concealment on Vision-Language Models

Evaluated models: Gemini 1.5 Pro, Gemini 2.5 Pro, GPT-4.5+3 more

Gemini 1.5 Pro
Gemini 2.5 Pro
GPT-4.5
GPT-4o
InternVL 2 8B
Qwen 2.5 VL 72B Instruct

Code-Mixed Phonetic Attack

Published May 20, 2025Analyzed Sep 7, 2025Source: arXiv

A vulnerability exists in multiple large language and multimodal models that allows for the bypass of safety filters through the use of code-mixed prompts with phonetic perturbations. An attacker can craft a prompt in a code-mixed language (e.g., Hinglish) and apply phonetic misspellings to sensitive keywords (e.g., spelling "hate" as "haet"). This technique causes the model's tokenizer to parse the sensitive word into benign sub-tokens, preventing safety mechanisms from flagging the harmful…

" Haet Bhasha aur Diskrimineshun": Phonetic Perturbations in Code-Mixed Hinglish to Red-Team LLMs

Evaluated models: Gemma 1.1 7B IT, GPT-4o, GPT-4o Mini+2 more

Gemma 1.1 7B IT
GPT-4o
GPT-4o Mini
Llama 3 8B Instruct
Mistral 7B Instruct v0.3

On-Device LLM Hijacking

Published May 19, 2025Analyzed Dec 9, 2025Source: arXiv

Mobile LLM agents utilizing vision-based screen perception (OCR or Multimodal Large Language Models) are vulnerable to Visual Prompt Injection via malicious GUI overlays. An attacker holding the SYSTEM_ALERT_WINDOW permission can deploy non-focusable floating windows (using FLAG_NOT_FOCUSABLE) containing adversarial text or fabricated UI elements over legitimate applications. Because the agent captures the entire screen buffer to interpret the device state, it ingests the adversarial overlay…

From Assistants to Adversaries: Exploring the Security Risks of Mobile LLM Agents

Evaluated models: GPT-4o

Universal VLLM Visual Bypass

Published May 2, 2025Analyzed Dec 9, 2025Source: arXiv

A vulnerability exists in Vision-Language Models (VLLMs) that allows for transferable, targeted adversarial attacks. Attackers can generate adversarial image perturbations using an ensemble of open-source surrogate models (primarily CLIP-based visual encoders) which effectively transfer to proprietary, black-box VLLMs. The attack leverages a specific optimization framework that combines a Visual Contrastive Loss with multiple positive/negative visual examples, rather than relying solely on…

Transferable Adversarial Attacks on Black-Box Vision-Language Models

Evaluated models: Qwen 2.5 VL 7B Instruct, Qwen 2.5 VL 72B Instruct, Llama 3.2 11B Vision Instruct+6 more

Qwen 2.5 VL 7B Instruct
Qwen 2.5 VL 72B Instruct
Llama 3.2 11B Vision Instruct
Llama 3.2 90B Vision Instruct
GPT-4o
GPT-4o Mini
Claude 3.5 Sonnet
Claude 3.7 Sonnet
Gemini 1.5 Pro

Memory Inception Jailbreak

Published Apr 29, 2025Analyzed Dec 30, 2025Source: arXiv

A contextual integrity vulnerability exists in the memory mechanisms of multi-turn Text-to-Image (T2I) generation systems (e.g., those integrating Large Language Models with diffusion models). The vulnerability, dubbed "Inception," arises because safety filters typically operate on a per-turn basis, inspecting only the current input prompt, while the image generation model operates on an aggregated context (memory) of the conversation history. An attacker can exploit this discrepancy by…

Inception: Jailbreak the memory mechanism of text-to-image generation systems

Evaluated models: GPT-3.5, GPT-4o, GPT-5+3 more

GPT-3.5
GPT-4o
GPT-5
DALL-E
Midjourney
Stable Diffusion

Chained Guardrail Bypass

Published Apr 18, 2025Analyzed Dec 30, 2025Source: arXiv

Large Language Model (LLM) agents operating in stateful environments (web browsers, operating systems, and tool-use contexts) are vulnerable to indirect prompt injection and multi-modal adversarial attacks. These vulnerabilities arise when agents process untrusted environmental observations—such as web accessibility trees, screen screenshots, or database query results—that contain concealed malicious instructions. Specifically, attackers can embed prompt injections into HTML accessibility…

DoomArena: A Framework for Testing AI Agents Against Evolving Security Threats

Evaluated models: GPT-4o, GPT-4o Mini, Claude 3.5 Sonnet+2 more

GPT-4o
GPT-4o Mini
Claude 3.5 Sonnet
Claude 3.7 Sonnet
Llama Guard

GUI Agent Fine-Print Injection

Published Apr 15, 2025Analyzed Dec 30, 2025Source: arXiv

LLM-powered GUI agents utilizing screenshot-based interpretation (such as those powered by GPT-4o, Claude 3.7 Sonnet, Gemini 2.0 Flash, and DeepSeek V3 0324) are vulnerable to Fine-Print Injection (FPI) and Deceptive Default (DD) attacks due to a lack of visual saliency filtering. Unlike human users who prioritize prominent UI elements, these agents perform "indiscriminate parsing," processing low-salience text (e.g., privacy policies, terms of service, footer disclaimers) with the same…

The Obvious Invisible Threat: LLM-Powered GUI Agents' Vulnerability to Fine-Print Injections

Evaluated models: GPT-4o, Gemini 2.0 Flash, Claude 3.7 Sonnet+2 more

GPT-4o
Gemini 2.0 Flash
Claude 3.7 Sonnet
Llama 3.3 70B Instruct
DeepSeek V3 0324

Metaphor-Based T2I Jailbreak

Published Mar 23, 2025Analyzed Apr 12, 2025Source: arXiv

A vulnerability in text-to-image (T2I) models allows bypassing safety filters through the use of metaphor-based adversarial prompts. These prompts, crafted using LLMs, indirectly convey sensitive content, exploiting the model's ability to infer meaning from figurative language while circumventing explicit keyword filters and model editing strategies.

Metaphor-based Jailbreaking Attacks on Text-to-Image Models

Evaluated models: DALL-E 3, Flux, Llama 3 8B Instruct+3 more

DALL-E 3
Flux
Llama 3 8B Instruct
Midjourney
Stable Diffusion v1.4
Stable Diffusion XL

Related research categories