The paper reports a reproducible black-box evaluation showing that adversarial user queries can cause deployed LLM applications to reveal hidden system prompts. In the authors’ measurement of 1,200 applications across six commercial platforms, 1,064 applications leaked prompt content (81.0%–93.5% per anonymized platform). This is a paper-reported result, not independently verified here. LeakBench and the official artifact repository provide defensive benchmark materials for controlled testing…
Understanding and Mitigating Prompt Leaking Attacks in Real-World LLM-Based Applications
OpenClaw is vulnerable to persistent memory poisoning, allowing an attacker to manipulate the agent's long-term memory store (MEMORY.md) via prompt injection. Because the autonomous agent continuously integrates this memory file as context for all subsequent reasoning and task planning, injected payloads act as durable behavioral constraints. This allows an attacker to persistently alter the agent's core policy, manipulate tool selection, and hijack future sessions without any further…
Taming openclaw: Security analysis and mitigation of autonomous llm agent threats
A vulnerability exists in Large Language Model (LLM) deployments and multi-agent systems where an autonomous attacker agent can systematically extract hidden system prompts through self-evolving interaction strategies. The vulnerability leverages a "JustAsk" framework which utilizes Upper Confidence Bound (UCB) exploration to dynamically select and refine attack vectors from a hierarchical taxonomy of 14 atomic skills (e.g., structural formatting, authority appeals) and 14 multi-turn…
Just Ask: Curious Code Agents Reveal System Prompts in Frontier LLMs
Evaluated models: o1, Llama 3.1 70B Hanami X1, Phi-4+38 more
A black-box guardrail reverse-engineering vulnerability exists in Large Language Model (LLM) serving systems that employ output filtering mechanisms. The vulnerability allows remote attackers to replicate the proprietary decision-making policy and rule sets of the target's safety guardrail without direct access to model parameters. This is achieved through a technique termed Guardrail Reverse-engineering Attack (GRA), which utilizes a reinforcement learning framework combined with genetic…
Large Language Models (LLMs), specifically variants of GPT-4o, DeepSeek-R1, OLMo-2, and Llama-4, are vulnerable to accelerated adaptive adversarial attacks due to excessive information leakage in observable output signals. When these models expose "thinking processes" (Chain-of-Thought traces) or token-level log-probabilities (logits) to the end user, they leak significant mutual information $I(Z;T)$ regarding the model's safety state or hidden instructions. This leakage allows adaptive attack…
Bits Leaked per Query: Information-Theoretic Bounds on Adversarial Attacks against LLMs
Evaluated models: DeepSeek R1, GPT-4o Mini 2024-07-18, Llama 4 Maverick 17B+4 more
A system prompt leakage vulnerability in GPT-4V allows extraction of internal system prompts through carefully crafted, incomplete conversations combined with image input. Extracted prompts can be used as highly effective jailbreak prompts, bypassing safety restrictions and leading to undesirable outputs, including revealing personally identifiable information from images.
Jailbreaking gpt-4v via self-adversarial attacks with system prompts