Impact

Integrity Research

Security research involving model-output integrity and reliability

539 matching entries

Integrity research results

539 entries

MAS Link Deception

Published Sep 1, 2025Analyzed Dec 9, 2025Source: arXiv

The evaluated MetaGPT multi-agent systems are vulnerable to "Web Fraud Attacks" due to insufficient semantic and structural validation of Uniform Resource Locators (URLs) by agentic models. A low-privilege compromised agent can exploit this vulnerability to induce other agents (including auditors and experts) into accepting, visiting, or processing malicious links. The vulnerability leverages the LLM's inability to distinguish between benign and malicious link structures when obfuscation…

Web fraud attacks against llm-driven multi-agent systems

Evaluated models: GPT-4o Mini, Gemini 2.5 Flash, DeepSeek Reasoner+1 more

GPT-4o Mini
Gemini 2.5 Flash
DeepSeek Reasoner
Llama 3 8B

Familiar Pattern Analysis Hijack

Published Aug 24, 2025Analyzed Feb 22, 2026Source: arXiv

Large Language Models (LLMs) utilized for static code analysis, code review, and autonomous software engineering exhibit a cognitive vulnerability termed "Abstraction Bias." When processing code that structurally resembles common algorithmic patterns (e.g., standard sorting algorithms, helper functions, or mathematical formulas), the model relies on high-level memorized representations of the algorithm's intent rather than analyzing the specific local logic. Adversaries can exploit this by…

Trust Me, I Know This Function: Hijacking LLM Static Analysis using Bias

Evaluated models: GPT-4o, Claude 3.5 Sonnet, Gemini 2.0 Flash+3 more

GPT-4o
Claude 3.5 Sonnet
Gemini 2.0 Flash
o3
Claude Sonnet 4
Gemini 2.5 Pro

LLM Agent TOCTOU Vulnerabilities

Published Aug 23, 2025Analyzed Aug 31, 2025Source: arXiv

A Time-of-Check to Time-of-Use (TOCTOU) vulnerability exists in LLM-enabled agentic systems that execute multi-step plans involving sequential tool calls. The vulnerability arises because plans are not executed atomically. An agent may perform a "check" operation (e.g., reading a file, checking a permission) in one tool call, and a subsequent "use" operation (e.g., writing to the file, performing a privileged action) in another tool call. A temporal gap between these calls, often used for LLM…

Mind the Gap: Time-of-Check to Time-of-Use Vulnerabilities in LLM-Enabled Agents

Evaluated models: GPT-4o

Multimodal Linking Visual Insecurity

Published Aug 21, 2025Analyzed Dec 9, 2025Source: arXiv

Multimodal Entity Linking (MEL) systems, encompassing both traditional dual-encoder models and Multimodal Large Language Models (MLLMs), are vulnerable to gradient-based white-box adversarial attacks. By applying imperceptible perturbations to visual inputs via Projected Gradient Descent (PGD), Auto-PGD (APGD), or Carlini & Wagner (CW) methods, an attacker can manipulate the visual embeddings generated by the model. This manipulation disrupts the cross-modal alignment structure, causing the…

On Evaluating the Adversarial Robustness of Foundation Models for Multimodal Entity Linking

Evaluated models: MiniGPT-4

Universal Prompt Disables Guardrails

Published Aug 18, 2025Analyzed Aug 31, 2025Source: arXiv

A universal prompt injection vulnerability, termed "Involuntary Jailbreak," affects multiple large language models. The attack uses a single prompt that instructs the model to learn a pattern from abstract string operators (X and Y). The model is then asked to generate its own examples of questions that should be refused (harmful questions) and provide detailed, non-refusal answers to them, in order to satisfy the learned operator logic. This reframes the generation of harmful content as a…

Involuntary Jailbreak

Evaluated models: Claude 3.5 Haiku, Claude 3.7 Sonnet, Claude Opus 4+29 more

Claude 3.5 Haiku
Claude 3.7 Sonnet
Claude Opus 4
Claude Opus 4.1
Claude Sonnet 4
Claude Sonnet 4.5
DeepSeek R1
DeepSeek R1 Distill Llama 70B
DeepSeek V3
Gemini 2.0 Flash
Gemini 2.5 Flash
Gemini 2.5 Flash-Lite
Gemini 2.5 Pro
Gemini 3 Pro
GPT-4.1
GPT-4.1 Mini
GPT-4o
GPT-oss 20B
Grok 3
Grok 3 Fast
Grok 3 Mini
Grok 4
Grok 4.1
Llama 3.1 8B
Llama 3.1 405B
Llama 3.3 70B
Llama 4 Scout
Llama 4 Maverick
Mistral Small 24B
Qwen 2.5 72B Instruct
Qwen 3 235B-A22B Instruct 2507
Qwen 3 Coder 480B-A35B Instruct

MDH: Hybrid Jailbreak Detection Strategy

Published Aug 14, 2025Analyzed Aug 31, 2025Source: arXiv

Large language models that support a developer role in their API are vulnerable to a jailbreaking attack that leverages malicious developer messages. An attacker can craft a developer message that overrides the model's safety alignment by setting a permissive persona, providing explicit instructions to bypass refusals, and using few-shot examples of harmful query-response pairs. This technique, named D-Attack, is effective on its own. A more advanced variant, DH-CoT, enhances the attack by…

Jailbreaking Commercial Black-Box LLMs with Explicitly Harmful Prompts

Evaluated models: GPT-3.5 Turbo, GPT-4o, GPT-4.1+13 more

GPT-3.5 Turbo
GPT-4o
GPT-4.1
o1-mini
o1
o3-mini
o3
o4-mini
Gemini 2.5 Pro
Gemini 2.0 Flash Thinking
Claude 3.5 Sonnet
Claude 3.7 Sonnet
Claude Sonnet 4
DeepSeek V3
DeepSeek R1 0528
DeepSeek R1

Back-Translation Watermark Stripping

Published Aug 11, 2025Analyzed Dec 30, 2025Source: arXiv

Implementations of Large Language Model (LLM) watermarking algorithms—specifically KGW (Kirchenbauer et al.), Semantic Invariant Robust (SIR) Watermark, Entropy-based Text Watermarking (EWD), and Unbiased Watermarking—are vulnerable to watermark stripping via adversarial text perturbation. When watermarked text generated by models such as OPT-1.3B is subjected to automated paraphrasing or back-translation (e.g., English $\to$ French $\to$ English), the embedded statistical signals are…

Signature vs. Substance: Evaluating the Balance of Adversarial Resistance and Linguistic Quality in Watermarking Large Language Models

Evaluated models: Llama 3 8B

Automated Red-Teaming Achieves 100% ASR

Published Aug 8, 2025Analyzed Aug 16, 2025Source: arXiv

Large Language Models (LLMs) are vulnerable to automated adversarial attacks that systematically combine multiple jailbreaking "primitives" into complex prompt chains. A dynamic optimization engine can generate and test billions of unique combinations of techniques (e.g., low-resource language translation, payload splitting, role-playing) to bypass safety guardrails. This combinatorial approach differs from manual red-teaming by systematically exploring the attack surface, achieving…

LLM Robustness Leaderboard v1--Technical report

Evaluated models: Yi Large, Qwen 2.5 72B Instruct, Qwen 2.5 7B Instruct+36 more

Yi Large
Qwen 2.5 72B Instruct
Qwen 2.5 7B Instruct
Qwen Max
Qwen Plus
Nova Lite v1
Nova Micro v1
Nova Pro v1
Claude 3 Haiku
Claude 3 Opus
Claude 3 Sonnet
Claude 3.5 Haiku 20241022
Claude 3.5 Sonnet 20240620
Claude 3.5 Sonnet 20241022
DeepSeek R1
DeepSeek V3
Gemini 1.5 Pro
Gemma 2 27B IT
Granite 3.1 8B
Llama 3.1 405B Instruct
Llama 3.1 70B Instruct
Llama 3.1 8B Instruct
Llama 3.2 11B Vision Instruct
Llama 3.2 1B Instruct
Llama 3.2 90B Vision Instruct
Llama 3.3 70B Instruct
Phi-4
Ministral 8B
Mistral Nemo
Mixtral 8x22B Instruct
Mixtral 8x7B Instruct
Pixtral Large 2411
GPT-4o 2024-08-06
GPT-4o 2024-11-20
GPT-4o Mini 2024-07-18
o1 2024-12-17
o3-mini 2025-01-14
Falcon 3 10B Instruct
Grok 2 1212

Latent Fusion Jailbreak Attack

Published Aug 8, 2025Analyzed Aug 31, 2025Source: arXiv

A vulnerability, known as Latent Fusion Jailbreak (LFJ), exists in certain Large Language Models that allows an attacker with white-box access to bypass safety alignments. The attack interpolates the internal hidden state representations of a harmful query and a thematically similar benign query. By using gradient-guided optimization to identify and modify influential layers and tokens, a fused hidden state is created that causes the model to generate prohibited content in response to the…

Latent Fusion Jailbreak: Blending Harmful and Harmless Representations to Elicit Unsafe LLM Outputs

Evaluated models: BERT, DeepSeek V3, GPT-3.5 Turbo+5 more

BERT
DeepSeek V3
GPT-3.5 Turbo
Guanaco 7B
Llama 2 7B Chat
Llama 3 70B
Mistral 7B Instruct
Vicuna 7B

Physical Patch Driving Hijack

Published Aug 7, 2025Analyzed Dec 9, 2025Source: arXiv

Multimodal Large Language Models (MLLMs) employed in autonomous driving (AD) systems are vulnerable to a physically realizable adversarial patch attack dubbed "PhysPatch." This vulnerability exists because MLLMs inherit susceptibility to visual adversarial perturbations from their vision backbones. The attack utilizes a semantic-aware mask initialization strategy combined with a potential field algorithm to identify physically plausible regions for patch placement within a driving scene (e.g…

PhysPatch: A Physically Realizable and Transferable Adversarial Patch Attack for Multimodal Large Language Models-based Autonomous Driving Systems

Evaluated models: LLaVA v1.6 13B, Qwen 2.5 VL 72B Instruct, Llama 3.2 90B Vision Instruct+8 more

LLaVA v1.6 13B
Qwen 2.5 VL 72B Instruct
Llama 3.2 90B Vision Instruct
GPT-4o
GPT-4.1
Claude Sonnet 4
Gemini 2.0 Flash
Qwen 2.5 VL Max
o3
Gemini 2.5 Flash
QVQ-Plus

Related research categories