Safety Vulnerabilities

Vision-Language Models (VLMs) are vulnerable to a universal and transferable adversarial attack dubbed "UltraBreak." This vulnerability allows remote attackers to bypass safety alignment filters and elicit harmful responses (e.g., hate speech, dangerous instructions) by supplying a single, specifically crafted adversarial image alongside a text query. Unlike traditional gradient-based attacks that optimize for specific token sequences (cross-entropy loss) and result in brittle…

Text scoring models, including dense retrievers, rerankers, and reward models, are vulnerable to score manipulation attacks via search-based discrete perturbations and content injection. An attacker can systematically modify candidate texts using rudimentary string manipulations, gradient-guided token swaps (e.g., HotFlip), masked language modeling (MLM) swaps, or query/sentence injections to spuriously increase model scores. This structural failure condition allows an irrelevant passage or a…

Vision Language Models (VLMs) utilizing independent vision encoders (e.g., ViT) and Large Language Model (LLM) decoders are vulnerable to Split-Image Visual Jailbreak Attacks (SIVA). The vulnerability arises from an architectural and alignment discrepancy: while the vision encoder processes image fragments (splits) in isolation via constrained attention or block-diagonal masks, the LLM decoder aggregates these features via cross-attention to reconstruct the semantic content. Current safety…

A vulnerability in Large Vision-Language Models (LVLMs) allows attackers to bypass safety guardrails via a Multi-Turn Adaptive Prompting Attack (MAPA). Instead of triggering safety mechanisms with an immediate, explicit malicious request, the attacker iteratively injects malicious intent across multiple conversation turns by alternating between text and visual modalities. At each turn, the attack dynamically tests three prompt configurations (unconnected text only, unconnected text + malicious…

Large Vision-Language Models (LVLMs) are vulnerable to Visual Memory Injection (VMI), a stealthy targeted attack targeting multi-turn conversations. An attacker can embed an imperceptible adversarial perturbation ($L_\infty \le 8/255$) into a seemingly benign image. Because the visual input persists in the model's context throughout a multi-turn dialogue, the injected payload remains dormant. By utilizing "benign anchoring" and "context-cycling" during optimization, the attacker ensures the…

Audio Large Language Models (ALLMs) integrated into voice agent systems for high-stakes domains (banking, IT support, logistics) are vulnerable to multimodal adversarial attacks via spoken interaction. Adversaries can exploit the model's inherent compliance and contextual awareness through multi-turn dialogue to bypass authentication safeguards, escalate privileges (e.g., unauthorized credit limit increases), exfiltrate sensitive Personally Identifiable Information (PII), and poison…

Self-evolving Large Language Model (LLM) agents that utilize long-term memory mechanisms (such as Vector Databases for Retrieval-Augmented Generation or Sliding Window buffers) are vulnerable to persistent indirect prompt injection. This vulnerability, termed "Zombie Agent," occurs when the agent's memory update function ($F_M$) processes attacker-controlled content retrieved from external sources (e.g., web pages, documents) and commits it to long-term storage without sufficient sanitization…

The paper reports a black-box jailbreak evaluation in which a ReAct-style loop adaptively rewrites unsafe text prompts and selectively applies blur, DCT filtering, or recoloring to image regions identified as safety-sensitive. The combined cross-modal strategy is intended to make harmful image-text requests appear less objectionable to a vision-language model while preserving enough semantics to elicit an answer. This is a specific, security-relevant evaluation, although the reported results…

The paper describes a specific black-box jailbreak evaluation, BVS, in which fragmented visual content is mixed with neutral imagery and paired with reconstruction-oriented text so harmful intent is only recomposed during multimodal reasoning. The authors report that this can bypass input and output safety assumptions in image-generating MLLMs. A safe defensive reproduction should use synthetic, non-harmful stand-ins for prohibited concepts, test whether fragmented cross-modal inputs are…

Conversational Large Language Model (LLM) agents integrated with privileged data sources (e.g., medical records, organizational emails) are vulnerable to contextually inappropriate information disclosure due to failures in enforcing Contextual Integrity (CI) norms. Standard semantic input/output filters and generic safety guardrails (e.g., Llama Guard) fail to detect "mosaic attacks" and multi-turn conversational manipulation. In these attacks, adversaries decompose a malicious query into a…