Impact

Safety Research

Security concerns affecting AI safety and alignment

798 matching entries

Safety research results

798 entries

MAS Pipeline Weak Links

Published 11/1/2025

Analyzed 1/14/2026

Centralized Multi-Agent Systems (MAS) architectures, specifically those implemented using frameworks such as Magentic-One, LangGraph, and OpenAI Agents, contain design-level vulnerabilities facilitating safety alignment bypasses. Two primary mechanisms drive this vulnerability: 1. Context Fragmentation via Atomic Delegation: The orchestrator decomposes adversarial prompts into atomic, granular sub-tasks (e.g., "click button," "write function header") before delegating them to sub-agents. These…

Exposing Weak Links in Multi-Agent Systems under Adversarial Prompting

Evaluated models: GPT-4o

Source: arXiv

Meta-Optimized LLM Judge Jailbreak

Published 11/1/2025

Analyzed 11/20/2025

A vulnerability in Large Language Models (LLMs) allows for systematic jailbreaking through a meta-optimization framework called AMIS (Align to MISalign). The attack uses a bi-level optimization process to co-evolve both the jailbreak prompts and the scoring templates used to evaluate them.

Align to Misalign: Automatic LLM Jailbreak with Meta-Optimized LLM Judges

Evaluated models: Claude 3.5 Haiku, Claude 3.5 Sonnet, Claude Sonnet 4 +3 more

Source: arXiv

Multi-Agent Multimodal Jailbreak

Published 11/1/2025

Analyzed 12/9/2025

The JPRO (Automated Multimodal Jailbreaking via Multi-Agent Collaboration) framework exploits a vulnerability in Large Vision-Language Models (VLMs) related to insufficient cross-modal safety alignment and lack of maliciousness sustainability in multi-turn dialogues. The attack leverages a multi-agent system (Planner, Attacker, Modifier, Verifier) to automate the generation of adversarial image-text pairs. By employing hybrid tactics—such as combining role-playing with malicious content…

JPRO: Automated Multimodal Jailbreaking via Multi-Agent Collaboration Framework

Evaluated models: GPT-4o, GPT-4o Mini, GPT-4.1 +3 more

Source: arXiv

Needle-in-Haystack Jailbreak

Published 11/1/2025

Analyzed 12/8/2025

A safety bypass vulnerability, dubbed "Ninja" (Needle-in-a-haystack jailbreak), exists in long-context Large Language Models (LLMs). The vulnerability exploits a degradation in safety alignment that occurs when a harmful goal is embedded within a massive, benign context window. Unlike traditional adversarial attacks that use unintelligible strings or "many-shot" attacks that use harmful examples, this method utilizes thematically relevant but innocuous text (the "haystack"). The attack…

Jailbreaking in the Haystack

Evaluated models: GPT-4o, Llama 3.1 8B Instruct, Gemini 2.0 Flash +2 more

Source: arXiv

Pervasive Multi-turn Jailbreaks

Published 11/1/2025

Analyzed 12/8/2025

Multiple open-weight Large Language Models (LLMs)—specifically those prioritizing capability over safety alignment—exhibit a critical vulnerability to adaptive multi-turn prompt injection and jailbreak attacks. While these models effectively reject isolated, single-turn adversarial inputs (averaging ~13.11% Attack Success Rate), they fail to maintain safety guardrails and policy enforcement across extended conversational contexts. By leveraging iterative strategies such as "Crescendo" (gradual…

Death by a Thousand Prompts: Open Model Vulnerability Analysis

Evaluated models: GPT-oss 20B, Llama 3.3 70B Instruct, Mistral Large 2 +5 more

Source: arXiv

Semantic Intention Obfuscation

Published 11/1/2025

Analyzed 12/8/2025

The KG-DF (Knowledge Graph Defense Framework) contains a logic vulnerability in its Semantic Parsing Module, specifically within the keyword extraction phase defined as $K_{core} = \text{LLM}(P_{prompt})$. The framework relies on a Large Language Model (e.g., GPT-3.5-turbo) to distill user input into keywords ($K_{core}$), which are then embedded to retrieve security warning triples ($T_{match}$) from a Knowledge Graph.

KG-DF: A Black-box Defense Framework against Jailbreak Attacks Based on Knowledge Graphs

Evaluated models: GPT-3.5, GPT-4, Llama 2 7B +1 more

Source: arXiv

Speech-Audio Composition Attack

Published 11/1/2025

Analyzed 12/30/2025

Multimodal Large Language Models (MLLMs) capable of processing speech and audio are vulnerable to Speech-Audio Compositional Attacks. This vulnerability exists because current safety mechanisms often rely on text-only transcription or fail to analyze the full acoustic context of an input. By manipulating the composition of audio signals, an attacker can bypass safety filters and elicit harmful responses. The attacks exploit three specific mechanisms: (1) Speech Overlap, where harmful…

Speech-Audio Compositional Attacks on Multimodal LLMs and Their Defense with SALMONN-Guard

Evaluated models: Qwen2-Audio 7B, Qwen 2.5 Omni 7B, Step-Audio 2 Mini Base +6 more

Source: arXiv

Template and Suffix Optimization

Published 11/1/2025

Analyzed 12/1/2025

A vulnerability exists in multiple Large Language Models (LLMs) that allows for safety alignment bypass through an advanced jailbreaking technique called Template and Suffix Optimization (TASO). The attack combines two distinct optimization methods in an alternating, iterative feedback loop. First, a semantically meaningless adversarial suffix is optimized (e.g., using gradient-based methods like GCG) to force the LLM to begin its response with an affirmative phrase (e.g., "Sure, here is...")…

TASO: Jailbreak LLMs via Alternative Template and Suffix Optimization

Evaluated models: Baichuan 2 13B, Baichuan 2 7B, DeepSeek 7B +27 more

Source: arXiv

Weak-OOD Jailbreak Boost

Published 11/1/2025

Analyzed 12/30/2025

Vision-Language Models (VLMs) are vulnerable to a jailbreak attack vector termed "weak-OOD" (weak Out-of-Distribution), specifically instantiated via the JOCR (Jailbreak via OCR-Aware Embedded Text Perturbation) method. The vulnerability arises from an asymmetry between the model's pre-training phase (which establishes robust OCR capabilities and intent perception) and the safety alignment phase (which lacks generalization to visual anomalies). Attackers can embed malicious text instructions…

Why does weak-OOD help? A Further Step Towards Understanding Jailbreaking VLMs

Evaluated models: GPT-4o, GPT-4o Mini, GPT-4.1 +3 more

Source: arXiv

PolyJailbreak Cross-Modal Safety Asymmetry

Published 10/20/2025

Analyzed 7/20/2026

The paper describes a reproducible black-box evaluation and attack framework, PolyJailbreak, for multimodal LLMs. It reports that uneven text-versus-vision safety alignment allows jointly optimized text and image inputs to bypass refusal behavior without model internals. The authors attribute this to visual alignment weakening textual refusal representations and to cross-modal fusion making harmful intent harder to separate from benign intent. These are paper-reported findings, not…

Multimodal Safety Is Asymmetric: Cross-Modal Exploits Unlock Black-Box MLLMs Jailbreaks

Evaluated models: LLaVA 1.5 7B, LLaVA 1.6 7B, Qwen-2.5-VL (7B) +5 more

Source: arXiv

Related research categories