Architecture Components

Multimodal Research

Security research involving multimodal AI systems

11 matching entries out of 176 in this category

Multimodal research results

11 entries

Mobile Agent Visual Spoofing

Published Feb 11, 2026Analyzed Feb 22, 2026Source: arXiv

Mobile Large Language Model (LLM) agents operating under the "Screen-as-Interface" paradigm are vulnerable to visual indirect prompt injection and state desynchronization. Agents that rely on unstructured visual data (screenshots) and Accessibility Service APIs to perceive the environment lack a mechanism to distinguish between trusted system UI elements and untrusted content (e.g., web pages, emails, or malicious overlays). An attacker can inject visual cues, fake notifications, or hidden…

Blind Gods and Broken Screens: Architecting a Secure, Intent-Centric Mobile Agent Operating System

Evaluated models: Not reported

Adaptive Multimodal Reasoning Jailbreaks

Published Jan 29, 2026Analyzed Jul 20, 2026Source: arXiv

The paper reports a black-box jailbreak evaluation in which a ReAct-style loop adaptively rewrites unsafe text prompts and selectively applies blur, DCT filtering, or recoloring to image regions identified as safety-sensitive. The combined cross-modal strategy is intended to make harmful image-text requests appear less objectionable to a vision-language model while preserving enough semantics to elicit an answer. This is a specific, security-relevant evaluation, although the reported results…

Jailbreaks on Vision Language Model via Multimodal Reasoning

Evaluated models: Gemini 2.0 Flash

Mobile Agent Channel Subversion

Published Oct 31, 2025Analyzed Dec 30, 2025Source: arXiv

Mobile LLM-based agents (including Mobile-Agent-E, AppAgent, AutoDroid, and others) are vulnerable to indirect prompt injection attacks delivered via untrusted third-party mobile channels, such as in-app advertisements, system notifications, and embedded webviews. These agents utilize Multimodal Large Language Models (MLLMs) to perceive the device state via screenshots or accessibility trees. The vulnerability exists because the agents concatenate the user's prompt ($p$) with the environmental…

Measuring the Security of Mobile LLM Agents under Adversarial Prompts from Untrusted Third-Party Channels

Evaluated models: GPT-3.5 Turbo, GPT-4 Turbo, GPT-4o+1 more

GPT-3.5 Turbo
GPT-4 Turbo
GPT-4o
GPT-4.1 Mini

Physical Agent Object Misdirection

Published Sep 20, 2025Analyzed Jan 14, 2026Source: arXiv

AdvEDM reveals a vulnerability in Vision-Language Model (VLM) based Embodied Decision-Making (EDM) systems, such as those used in autonomous driving and robotic manipulation. The vulnerability allows an attacker to launch fine-grained adversarial attacks that selectively modify the perception of specific objects in an input image—either by removing them (Semantic Removal) or adding them (Semantic Addition)—while preserving the semantic integrity of the rest of the scene.

AdvEDM: Fine-grained Adversarial Attack against VLM-based Embodied Agents

Evaluated models: BLIP-2, MiniGPT-4, LLaVA-v2+5 more

BLIP-2
MiniGPT-4
LLaVA-v2
Otter-Image
OpenFlamingo-v2
Dolphins
EmbodiedGPT
GPT-4o

GUI Agent Dark Pattern Blindness

Published Sep 12, 2025Analyzed Dec 9, 2025Source: arXiv

Large Language Model (LLM)-powered GUI agents exhibit a vulnerability to deceptive interface designs (dark patterns) due to goal-driven optimization and procedural myopia. When executing natural language instructions on web interfaces, these agents consistently prioritize minimizing steps and achieving task completion over user safety or privacy. Agents frequently recognize manipulative elements—such as pre-selected consent checkboxes, hidden costs, or trick questions—in their internal…

Dark Patterns Meet GUI Agents: LLM Agent Susceptibility to Manipulative Interfaces and the Role of Human Oversight

Evaluated models: GPT-4o, Claude 3.7 Sonnet, DeepSeek V3+1 more

GPT-4o
Claude 3.7 Sonnet
DeepSeek V3
Gemini 2.0 Flash

Patching Induces Emergent Deception

Published Jul 12, 2025Analyzed Dec 30, 2025Source: arXiv

Adversarial Activation Patching enables the induction of emergent deceptive behaviors in safety-aligned transformer-based Large Language Models (LLMs). By extracting intermediate activations ($A_{d}$) generated during the processing of a deceptive or harmful prompt and injecting them into the forward pass of a benign target prompt ($x_{t}$) at specific layers (specifically mid-layers, e.g., 5-10 in 32-layer architectures), an attacker can manipulate the model's internal reasoning circuits…

Adversarial activation patching: A framework for detecting and mitigating emergent deception in safety-aligned transformers

Evaluated models: GPT-4

Memory Inception Jailbreak

Published Apr 29, 2025Analyzed Dec 30, 2025Source: arXiv

A contextual integrity vulnerability exists in the memory mechanisms of multi-turn Text-to-Image (T2I) generation systems (e.g., those integrating Large Language Models with diffusion models). The vulnerability, dubbed "Inception," arises because safety filters typically operate on a per-turn basis, inspecting only the current input prompt, while the image generation model operates on an aggregated context (memory) of the conversation history. An attacker can exploit this discrepancy by…

Inception: Jailbreak the memory mechanism of text-to-image generation systems

Evaluated models: GPT-3.5, GPT-4o, GPT-5+3 more

GPT-3.5
GPT-4o
GPT-5
DALL-E
Midjourney
Stable Diffusion

Agent System Orchestration Hijack

Published Mar 15, 2025Analyzed Jan 14, 2026Source: arXiv

Multi-agent systems (MAS) utilizing Large Language Model (LLM) orchestration are vulnerable to control-flow hijacking via indirect prompt injection, leading to Remote Code Execution (RCE). This vulnerability arises when a sub-agent (e.g., a file surfer or web surfer) processes untrusted input containing adversarial metadata, such as simulated error messages or administrative instructions. The sub-agent faithfully reproduces this adversarial content in its report to the orchestrator agent. The…

Multi-agent systems execute arbitrary malicious code

Evaluated models: GPT-4o, GPT-4o Mini, Gemini 1.5 Pro+1 more

GPT-4o
GPT-4o Mini
Gemini 1.5 Pro
Gemini 1.5 Flash

Flowchart-based LVLM Jailbreak Attack

Published Feb 28, 2025Analyzed Mar 19, 2025Source: arXiv

FC-Attack leverages automatically generated flowcharts containing step-by-step descriptions derived or rephrased from harmful queries, combined with a benign textual prompt, to jailbreak Large Vision-Language Models (LVLMs). The vulnerability lies in the model's susceptibility to visual prompts containing harmful information within the flowcharts, thus bypassing safety alignment mechanisms.

FC-Attack: Jailbreaking Large Vision-Language Models via Auto-Generated Flowcharts

Evaluated models: Claude 3.5 Sonnet 20240620, Gemini 1.5 Flash, GPT-4o 2024-08-06+4 more

Claude 3.5 Sonnet 20240620
Gemini 1.5 Flash
GPT-4o 2024-08-06
GPT-4o Mini 2024-07-18
InternVL2.5 8B
LLaVA NeXT 8B
Qwen2-VL 7B Instruct

AD Black-Box Cascading Disruption

Published Jan 23, 2025Analyzed Dec 9, 2025Source: arXiv

Vision Language Models (VLMs) integrated into autonomous driving (AD) systems are vulnerable to a black-box adversarial attack method termed Cascading Adversarial Disruption (CAD). The vulnerability stems from the model's susceptibility to optimized visual perturbations that disrupt the decision-making reasoning chain (perception, prediction, and planning). Attackers can generate adversarial images or physical patches by aligning visual noise with deceptive textual semantics in the model's…

Black-box adversarial attack on vision language models for autonomous driving

Evaluated models: GPT-4, GPT-4o, InstructBLIP

Related research categories