Attack Context

AI agents Research

Security research involving AI agents and autonomous systems

8 matching entries out of 267 in this category

AI agents research results

8 entries

VLM E-commerce Attack Surface

Published Mar 17, 2026Analyzed Apr 11, 2026Source: arXiv

LLaVA-v1.5-7B, when deployed as a vision-language autonomous agent, is highly vulnerable to adversarial image perturbations. An attacker can inject imperceptibly modified images into a web environment (such as an e-commerce storefront). When the VLM agent captures a screenshot containing the perturbed image, the visual noise forces the model to misclassify the scene and output incorrect, structured JSON actions. This allows an attacker to hijack the agent's task execution, bypassing the user's…

Adversarial attacks against Modern Vision-Language Models

Evaluated models: Qwen 2.5 VL 7B Instruct, LLaVA 1.5 7B

Video-LLM Token Flood

Published Mar 2, 2026Analyzed Mar 9, 2026Source: arXiv

Video-based Large Language Models (Video-LLMs) are vulnerable to a universal Energy-Latency Attack (ELA) that triggers a Denial-of-Service (DoS) via spatially concentrated adversarial patches. Because video architectures rely on temporal subsampling and pooling which act as a low-pass filter against full-frame diffuse noise, an attacker can bypass this compression by anchoring cross-modal attention to a dense, localized visual anomaly. By injecting a fixed, content-agnostic patch into the…

VidDoS: Universal Denial-of-Service Attack on Video-based Large Language Models

Evaluated models: LLaVA-NeXT-Video 7B, Qwen3-VL 4B Instruct, Video-LLaVA 7B

Visual Compression Mismatch Attack

Published Jan 29, 2026Analyzed Feb 22, 2026Source: arXiv

A vulnerability exists in Large Vision-Language Models (LVLMs) utilizing visual token compression mechanisms (e.g., VisionZip, VisPruner) to reduce inference latency. The vulnerability stems from an optimization-inference mismatch where standard adversarial defenses assume full-token processing, while the deployed model utilizes a subset of tokens selected via importance metrics (typically attention scores).

On the Adversarial Robustness of Large Vision-Language Models under Visual Token Compression

Evaluated models: LLaVA 1.5 7B

Embodied Cross-Modal Misalignment

Published Nov 20, 2025Analyzed Jan 14, 2026Source: arXiv

OpenVLA, a Vision-Language-Action (VLA) model, contains a vulnerability regarding multimodal adversarial robustness. The model lacks sufficient cross-modal alignment stability, allowing attackers to disrupt the grounding between visual perception and linguistic instructions. By utilizing the "VLA-Fool" framework, adversaries can inject perturbations via three vectors: (1) Semantically Greedy Coordinate Gradient (SGCG), which alters specific linguistic tokens (referential cues, attributes…

When alignment fails: Multimodal adversarial attacks on vision-language-action models

Evaluated models: Not reported

Airborne Audio LLM Hijack

Published Jul 7, 2025Analyzed Jan 14, 2026Source: arXiv

Audio-based Large Language Models (ALLMs), specifically Qwen2-Audio, are vulnerable to over-the-air adversarial audio attacks. An attacker with white-box access can generate robust adversarial audio perturbations using gradient-based optimization combined with audio augmentation techniques (specifically SpecAugment, translation, and additive noise). These perturbations, when played through a speaker in the physical environment, manipulate the ALLM processing the audio via a microphone. This…

Attacker's Noise Can Manipulate Your Audio-based LLM in the Real World

Evaluated models: Not reported

VLM Navigation Object Fusion

Published May 29, 2025Analyzed Dec 30, 2025Source: arXiv

The Vision-Language Model (VLM) perception module in Vision-and-Language Navigation (VLN) agents is vulnerable to adversarial 3D object injection via the Adversarial Object Fusion (AdvOF) framework. An attacker can generate physically plausible 3D objects with adversarial perturbations capable of deceiving the agent's VLM across multiple viewing angles and distances. The vulnerability exists due to a misalignment between 3D physical manipulations and the agent's 2D image perception, combined…

Disrupting Vision-Language Model-Driven Navigation Services via Adversarial Object Fusion

Evaluated models: Not reported

Image-Text Jailbreak

Published Mar 5, 2025Analyzed Mar 8, 2026Source: arXiv

Multimodal Large Language Models (MLLMs) are vulnerable to coupled cross-modal jailbreak attacks that combine continuous visual perturbations with discrete textual manipulations. Because standard alignment and single-modality defenses (such as text-only safety tuning or isolated vision-encoder adversarial training) fail to secure the cross-modal interaction, attackers can simultaneously apply gradient-based noise (e.g., PGD) to input images and adversarial suffixes (e.g., GCG) to text prompts…

E2AT: Multimodal Jailbreak Defense via Dynamic Joint Optimization for Multimodal Large Language Models

Evaluated models: LLaVA 1.5 7B, Bunny 1.0 4B, Mplug-owl2

Visual Modality Jailbreak

Published May 30, 2024Analyzed Dec 28, 2024Source: arXiv

A vulnerability in multimodal large language models (MLLMs) allows for efficient jailbreaking attacks by leveraging visual input to bypass safety mechanisms. The attack constructs a multimodal model by adding a visual module to the target LLM, then uses a modified PGD algorithm to optimize visual input to generate jailbreaking embeddings. These embeddings are then converted back into text and appended to harmful queries, successfully eliciting objectionable content from the target LLM.

Efficient LLM-Jailbreaking by Introducing Visual Modality

Evaluated models: ChatGLM 6B, GPT-3.5 Turbo, Mistral 7B

Related research categories