Sleeper Memory Poisoning in LLM Agents

Description

The paper presents a specific black-box indirect prompt-injection evaluation: attacker-controlled external content can cause a memory-enabled assistant or external memory manager to persist a fabricated user memory, which may later be retrieved in a separate session and steer responses or agent actions. The authors evaluate injection, retrieval, and conditional adversarial usage separately across synthetic document and future-session datasets. The released repository provides defensive benchmark and smoke configurations; safe reproduction should use synthetic memories and sandboxed or mocked actions only.

Examples

See the primary source for the paper's evaluation and defensive reproduction context: https://arxiv.org/abs/2605.15338 (opens in a new tab)

Impact

A poisoned memory can persist after the original untrusted content is gone, creating delayed, repeated integrity and safety failures in later conversations. Paper-reported results include tool-based behavioral-memory injection rates up to 99.8% for GPT-5.5 and 95.0% for Kimi-K2.6. For goal-adjacent agent-action evaluations, reported retrieval rates were 94–98%, and conditional adversarial-usage rates were 60–89% across evaluated models. These are controlled, paper-reported measurements, not independently verified production rates.

Affected Systems

• LLM assistants with persistent cross-session memory
• Agents whose model can invoke a memory-writing tool
• Systems using a separate LLM memory manager
• Memory retrieval pipelines using semantic similarity, LLM selection, or all-memories-in-context

Mitigation Steps

• Treat documents, webpages, emails, repositories, and other external context as untrusted; prevent them from directly authorizing persistent memory writes.
• Require explicit user confirmation for memory writes derived from external content, and display the proposed memory plus its provenance before persistence.
• Store source provenance and trust labels with memories; prevent untrusted memories from becoming authoritative instructions or influencing high-impact actions.
• Apply system-prompt hardening and untrusted-content delimiters, while testing against adaptive attacks because the paper reports uneven robustness across models.
• Scan external documents for memory-poisoning instructions before processing and quarantine or remove suspicious spans.
• Gate consequential tool use on current-session user intent rather than retrieved memories alone; require confirmation for sensitive actions.
• Provide user-visible memory review, correction, and deletion controls, and periodically audit stored memories for unsupported or suspicious entries.

Evidence

• https://arxiv.org/html/2605.15338 (opens in a new tab) — Sections 3.2.1–3.2.2 define a black-box external-content adversary and the three-stage injection, retrieval, and adversarial-usage success condition.
• https://arxiv.org/html/2605.15338 (opens in a new tab) — Sections 5.1–5.4 describe the tool-based and external-manager memory regimes, separate future-session evaluation, retrieval mechanisms, and IR/RR/AUR metrics.
• https://arxiv.org/html/2605.15338 (opens in a new tab) — Section 6.1 and Table 1 report evaluation of GPT-5.4, GPT-5.5, Claude Sonnet 4.6, Gemini 3.1 Pro, Kimi-K2.6, and DeepSeek V4-Pro, including up to 99.8% tool-based behavioral injection rate.
• https://arxiv.org/html/2605.15338 (opens in a new tab) — Section 6.2 and Table 2 report goal-adjacent agent-action retrieval rates of 94–98% and conditional adversarial-usage rates of 60–89%; the paper states AUR is conditional on retrieval.
• https://arxiv.org/html/2605.15338 (opens in a new tab) — Section 6.3 and Appendix J evaluate prompt hardening, untrusted-content spotlighting, hybrid defenses, and document scanning; Section 6.3 reports defenses remain brittle for some models and adaptive attacks.
• https://arxiv.org/html/2605.15338 (opens in a new tab) — Appendix A states limitations: LLM-judge scoring, partially observable provider memory pipelines, and no systematic evaluation of deletion, correction, user review, or provenance-aware editing.
• https://github.com/ivaxi0s/LLM-agent-memory-poisoning (opens in a new tab) — directly linked paper artifact containing the public evaluation framework, released datasets, smoke configurations, and dry-run guidance for reproducing the paper’s measurements.