Language Model Security Database

The paper describes GhostWriter, a reproducible two-phase attack against tool-using agents with persistent memory: untrusted email or calendar content is admitted into long-term memory, then later retrieved during a benign user task and treated as trusted context. In the authors’ controlled evaluation, this could steer subsequent agent actions despite the adversary lacking direct access to the agent, memory store, account, or later prompt. The paper reports an average 98% memory-injection rate…

The paper describes a reproducible black-box multimodal jailbreak evaluation, INFER/INFER+, in which dense image typography, nested cross-modal references, recursive visual layouts, and entropy-guided search increase processing complexity and weaken refusal behavior in large vision-language models. The authors report average ASRs of 88.6% on open-source models and 84.0% on commercial models; these are paper-reported measurements, not independently verified facts. For safe defensive…

The paper reports a reproducible black-box evaluation showing that adversarial user queries can cause deployed LLM applications to reveal hidden system prompts. In the authors’ measurement of 1,200 applications across six commercial platforms, 1,064 applications leaked prompt content (81.0%–93.5% per anonymized platform). This is a paper-reported result, not independently verified here. LeakBench and the official artifact repository provide defensive benchmark materials for controlled testing…

The paper presents a concrete, reproducible security evaluation in which attacker-controlled instructions embedded in retrieved external content steer stateful, tool-calling LLM agents toward unauthorized actions. It adapts white-box GCG and black-box TAP to AgentDojo and evaluates single-task and task-universal attacks across 80 task pairs in four domains. The reported results show that semantic black-box optimization can discover functional prompt injections more effectively than…

MLingualFC is a reproducible black-box safety evaluation showing that harmful instructions rendered as multilingual flowchart images can bypass vision-language model safeguards more often than equivalent text-only inputs. The paper evaluates horizontal, vertical, and tortuous layouts across English, Hindi, Punjabi, Spanish, Romanian, and German. Reported results vary substantially by language, script, layout, and model; these are paper-reported measurements, not independently verified…

The paper describes and evaluates a reproducible application-layer weakness in agents with persistent memory: untrusted external content can cross the memory-write boundary, be stored as trusted factual, experience, or procedural memory, and influence later sessions. It identifies four write channels—explicit writes, policy-driven writes, compaction, and experience-to-procedure synthesis—and six attack classes. For safe defensive testing, use MPBench’s two-phase structure in an isolated agent…

SilentRetrieval describes a specific RAG corpus-integrity vulnerability: an attacker able to add a topically relevant document to a retrieval corpus can make that document rank highly and influence the generated answer while remaining fluent enough to evade simple perplexity checks. The paper evaluates a two-stage method combining retrieval-oriented document optimization with context-adaptive claim integration. A safe defensive reproduction is to use only isolated benchmark corpora and inert…

The paper presents a specific black-box indirect prompt-injection evaluation: attacker-controlled external content can cause a memory-enabled assistant or external memory manager to persist a fabricated user memory, which may later be retrieved in a separate session and steer responses or agent actions. The authors evaluate injection, retrieval, and conditional adversarial usage separately across synthetic document and future-session datasets. The released repository provides defensive…

M3Att demonstrates a reproducible knowledge-poisoning issue in medical multimodal RAG: an attacker with limited corpus-distribution knowledge can insert paired image-text entries whose visually perturbed images are broadly retrieved and whose clinically plausible misinformation steers downstream generation. The paper evaluates both white-box and black-box retrieval optimization and reports degraded diagnostic and report-generation utility across multiple datasets, retrievers, and LVLMs. These…

The paper reports a reproducible black-box jailbreak evaluation in which an evolutionary search generates simulated, role-labeled multi-turn dialogue histories and submits them as a single request. Five semantic mutators and graded judge feedback optimize conversational priming that can induce harmful responses despite direct-request refusals. Results are paper-reported and not independently verified.