Impact

Data security Research

Issues affecting data confidentiality and integrity

44 matching entries out of 171 in this category

Data security research results

44 entries

Black-Box System Prompt Leakage in Real-World LLM Applications

Published Jun 17, 2026Analyzed Jul 20, 2026Source: arXiv

The paper reports a reproducible black-box evaluation showing that adversarial user queries can cause deployed LLM applications to reveal hidden system prompts. In the authors’ measurement of 1,200 applications across six commercial platforms, 1,064 applications leaked prompt content (81.0%–93.5% per anonymized platform). This is a paper-reported result, not independently verified here. LeakBench and the official artifact repository provide defensive benchmark materials for controlled testing…

Understanding and Mitigating Prompt Leaking Attacks in Real-World LLM-Based Applications

Evaluated models: Llama 2 7B Chat, Llama 3.1 8B Instruct, Mistral 7B Instruct v0.3+5 more

Llama 2 7B Chat
Llama 3.1 8B Instruct
Mistral 7B Instruct v0.3
Qwen 3 4B Instruct 2507
Qwen 3 32B
Qwen 2.5 72B Instruct
Llama 3.3 70B Instruct
Qwen 3 30B-A3B Instruct-2507

Instruction Serialization Leak

Published Apr 1, 2026Analyzed Apr 10, 2026Source: arXiv

Large Language Models (LLMs) are vulnerable to system instruction leakage when extraction requests are framed as benign formatting, encoding, or structured-output tasks. While standard alignment and refusal mechanisms successfully block direct queries for system instructions, they fail when attackers request the instructions to be rendered in alternate representations (e.g., YAML, TOML, Base64, or system logs). The model's safety filters misinterpret the request as a harmless transformation or…

Automated Framework to Evaluate and Harden LLM System Instructions against Encoding Attacks

Evaluated models: GPT-4.1 Mini, GPT-3.5 Turbo, Gemini 2.5 Flash+1 more

GPT-4.1 Mini
GPT-3.5 Turbo
Gemini 2.5 Flash
Llama 3 8B

Stage-Sequential Agent Escalation

Published Mar 18, 2026Analyzed Apr 10, 2026Source: arXiv

Agentic Large Language Model (LLM) systems utilizing persistent memory, Retrieval-Augmented Generation (RAG) pipelines, and external tool connectors are vulnerable to Logic-layer Prompt Control Injection (LPCI). An attacker can inject obfuscated (e.g., encoded, structurally nested, or semantically reframed) payloads into external memory stores or RAG documents. These payloads bypass conventional inference-time plaintext content filters, persist across session boundaries, and remain dormant…

LAAF: Logic-layer Automated Attack Framework A Systematic Red-Teaming Methodology for LPCI Vulnerabilities in Agentic Large Language Model Systems

Evaluated models: GPT-4o Mini, Claude 3 Haiku, Llama 3.1 70B Instruct+2 more

GPT-4o Mini
Claude 3 Haiku
Llama 3.1 70B Instruct
Gemini 2.0 Flash
Mixtral 8x7B Instruct

Agent Lifecycle Compound Threats

Published Mar 12, 2026Analyzed Apr 10, 2026Source: arXiv

OpenClaw is vulnerable to persistent memory poisoning, allowing an attacker to manipulate the agent's long-term memory store (MEMORY.md) via prompt injection. Because the autonomous agent continuously integrates this memory file as context for all subsequent reasoning and task planning, injected payloads act as durable behavioral constraints. This allows an attacker to persistently alter the agent's core policy, manipulate tool selection, and hijack future sessions without any further…

Taming openclaw: Security analysis and mitigation of autonomous llm agent threats

Evaluated models: Not reported

Arbitrary Agent Topology Breach

Published Mar 11, 2026Analyzed Apr 10, 2026Source: arXiv

A vulnerability in LLM-based Multi-Agent Systems (LLM-MAS) allows an attacker who controls a single arbitrary agent to map and extract the system's entire confidential communication topology. Unlike prior attacks that rely on direct identity queries and administrative privileges, this attack infers topology stealthily purely from contextual and linguistic signals (stylometry, role-specific syntax), bypassing standard keyword-based and identity-filtering defenses. The exploit relies on a…

WebWeaver: Breaking Topology Confidentiality in LLM Multi-Agent Systems with Stealthy Context-Based Inference

Evaluated models: Llama 3 70B, Llama 3.1 8B, Mistral Large 12B+3 more

Llama 3 70B
Llama 3.1 8B
Mistral Large 12B
Qwen 2.5 7B
Gemma 2 9B
Phi-3

Personalized Agent Double Agent

Published Feb 9, 2026Analyzed Mar 9, 2026Source: arXiv

OpenClaw is vulnerable to Indirect Prompt Injection (IPI), Tool-Return Manipulation, and Persistent Memory Poisoning. The agent incorporates untrusted external content (e.g., fetched web pages) and external tool outputs directly into its observation stream without sufficient isolation. An attacker can embed malicious payloads into these external channels to hijack the agent's planning and execution trace. This allows the attacker to silently trigger high-privilege actions via OpenClaw's Skills…

From Assistant to Double Agent: Formalizing and Benchmarking Attacks on OpenClaw for Personalized Local AI Agent

Evaluated models: GPT-4o, Llama 3.1 70B, Qwen 2.5 7B

Voice Agent Behavioral Bypass

Published Feb 7, 2026Analyzed Feb 21, 2026Source: arXiv

Audio Large Language Models (ALLMs) integrated into voice agent systems for high-stakes domains (banking, IT support, logistics) are vulnerable to multimodal adversarial attacks via spoken interaction. Adversaries can exploit the model's inherent compliance and contextual awareness through multi-turn dialogue to bypass authentication safeguards, escalate privileges (e.g., unauthorized credit limit increases), exfiltrate sensitive Personally Identifiable Information (PII), and poison…

Aegis: Towards Governance, Integrity, and Security of AI Voice Agents

Evaluated models: GPT-4o, GPT-4o Mini, Gemini 1.5 Pro+4 more

GPT-4o
GPT-4o Mini
Gemini 1.5 Pro
Gemini 2.5 Flash
Gemini 2.5 Pro
Qwen2-Audio 7B
Qwen 2.5 Omni 7B

Autonomous Agent Prompt Reveal

Published Jan 29, 2026Analyzed Feb 21, 2026Source: arXiv

A vulnerability exists in Large Language Model (LLM) deployments and multi-agent systems where an autonomous attacker agent can systematically extract hidden system prompts through self-evolving interaction strategies. The vulnerability leverages a "JustAsk" framework which utilizes Upper Confidence Bound (UCB) exploration to dynamically select and refine attack vectors from a hierarchical taxonomy of 14 atomic skills (e.g., structural formatting, authority appeals) and 14 multi-turn…

Just Ask: Curious Code Agents Reveal System Prompts in Frontier LLMs

Evaluated models: o1, Llama 3.1 70B Hanami X1, Phi-4+38 more

o1
Llama 3.1 70B Hanami X1
Phi-4
Aion 1.0
Sonar Pro
Command A
Llama 4 Maverick
Llama 3.1 Nemotron Ultra 253B v1
Qwen 3 235B-A22B
Mercury
Hunyuan A13B Instruct
UI-TARS 1.5 7B
GPT-oss 120B
Jamba Mini 1.7
Hermes 4 70B
Step 3
LongCat Flash Chat
Tongyi DeepResearch 30B-A3B
Cydonia 24B v4.1
ERNIE 4.5 21B-A3B Thinking
Granite 4.0 H Micro
LFM2 8B-A1B
Nova Premier v1
Kimi K2 Thinking
KAT Coder Pro
Cogito v2.1 671B
Gemini 3 Pro Preview
Grok 4.1 Fast
Claude Opus 4.5
TNG R1T Chimera
Intellect 3
DeepSeek V3.2 Speciale
Trinity Mini
Mistral Large 2512
DeepSeek V3.1 Nex N1
MiMo V2 Flash
GLM-4.7
MiniMax M2.1
Seed 1.6
Molmo 2 8B
GPT-5.2 Codex

MCP Server-Side Injection

Published Jan 24, 2026Analyzed Feb 22, 2026Source: arXiv

The Model Context Protocol (MCP) specification v1.0 contains fundamental architectural vulnerabilities enabling server-side prompt injection and privilege escalation. The protocol relies on bidirectional sampling (sampling/createMessage) without cryptographic origin authentication or UI distinction, allowing connected servers to inject content that the LLM backend interprets as legitimate user input. Additionally, the protocol lacks isolation boundaries between concurrent server connections…

Breaking the Protocol: Security Analysis of the Model Context Protocol Specification and Prompt Injection Vulnerabilities in Tool-Integrated LLM Agents

Evaluated models: GPT-4o, Claude 3.5 Sonnet, Llama 3.1 70B

Production LLM Copyright Extraction

Published Jan 6, 2026Analyzed Feb 21, 2026Source: arXiv

Production Large Language Models (LLMs) are vulnerable to long-form training data extraction via a two-phase prompt injection attack. This vulnerability allows an attacker to recover substantial portions of memorized, copyrighted text (such as novels) by exploiting the model's autoregressive text completion capabilities. The attack methodology involves two distinct phases: 1. Prefix Completion Probe: The attacker provides a short "seed" sequence (e.g., the first sentence of a book) coupled…

Extracting Books from Production Language Models

Evaluated models: Claude 3.7 Sonnet 20250219, GPT-4.1 2025-04-14, Gemini 2.5 Pro+1 more

Claude 3.7 Sonnet 20250219
GPT-4.1 2025-04-14
Gemini 2.5 Pro
Grok 3

Related research categories