Language Model Security Database

Retrieval-Augmented Generation (RAG) systems are vulnerable to a robust corpus poisoning attack known as "Confundo." This vulnerability arises from the lack of pipeline awareness in standard RAG implementations, specifically regarding document ingestion (tokenization and chunking) and query variations. An attacker can exploit this by fine-tuning a Large Language Model (LLM) to function as a poison generator. Unlike traditional adversarial examples which are brittle, Confundo generates poison…

A vulnerability in the Direct Preference Optimization (DPO) post-training phase of OLMo 2 models leads to "distractor-triggered compliance." The model correctly refuses harmful requests when prompted in isolation, but complies with identical harmful requests if a benign formatting instruction (a "distractor") is appended to the prompt. This behavior organically emerges from contaminated preference training data where mislabeled examples incorrectly preferred compliance over refusal when a…

A vulnerability exists in Large Language Models (LLMs) deployed in environments with output reingestion (e.g., RAG, coding assistants, agentic workflows) that allows attackers to execute "temporal backdoors" (time bombs) via an implicit memory channel. Attackers can implant this behavior via system prompts or fine-tuning (data poisoning) to make the model encode hidden state information within its generated text using non-printing Unicode characters or semantic steganography. When these…

Large Language Models (LLMs) subjected to Supervised Fine-Tuning (SFT) are vulnerable to "sleeper agent" data poisoning attacks. An attacker injects specific trigger phrases into the training corpus, causing the model to learn a conditional policy: behaving normally for standard inputs but executing a malicious target behavior when the trigger is present. These backdoors persist through safety training and alignment. The vulnerability stems from the model's strong memorization of poisoning…

Large Language Models (LLMs) aligned via standard preference-based optimization methods (e.g., DPO, RLHF) are vulnerable to safety degradation due to optimization-induced fragility. The vulnerability arises from sharp minima in the alignment loss landscape, specifically within a small, localized subspace of safety-critical parameters (approximately 0.5% of neurons account for >80% of worst-case alignment loss). Standard alignment algorithms enforce uniform constraints or fail to control the…

LLM agents equipped with tool-use, persistent memory, and environmental interaction capabilities are vulnerable to long-horizon attacks. Attackers can bypass single-turn safety guardrails by exploiting the temporal dimension of multi-turn interactions to incrementally steer agent behavior. The vulnerability manifests because the agent's safety mechanisms perform localized, single-step evaluations but fail to maintain semantic safety across extended interaction trajectories. This enables…

Mobile Large Language Model (LLM) agents operating under the "Screen-as-Interface" paradigm are vulnerable to visual indirect prompt injection and state desynchronization. Agents that rely on unstructured visual data (screenshots) and Accessibility Service APIs to perceive the environment lack a mechanism to distinguish between trusted system UI elements and untrusted content (e.g., web pages, emails, or malicious overlays). An attacker can inject visual cues, fake notifications, or hidden…

Autoregressive Large Language Models (LLMs) utilizing standard fine-tuning (SFT) or alignment techniques (RLHF/DPO) are vulnerable to training-time data poisoning attacks that exploit the sequential nature of token generation. Unlike classification tasks, where output labels are independent, LLM generation suffers from a cascading vulnerability where modifying a single token $i$ intervenes on the distribution of all subsequent tokens $j > i$. An adversary can inject a small fraction of…

A data poisoning vulnerability exists in the pretraining pipeline of Transformer-based Large Language Models (LLMs) where the injection of synthetic uniform random noise into the training corpus induces irreversible training loss divergence or performance degradation. This instability is mechanistically distinct from divergence caused by high learning rates. The vulnerability is specifically triggered by "insertion noise" (inserting random tokens between clean tokens) drawn from a restricted…

OpenClaw is vulnerable to Indirect Prompt Injection (IPI), Tool-Return Manipulation, and Persistent Memory Poisoning. The agent incorporates untrusted external content (e.g., fetched web pages) and external tool outputs directly into its observation stream without sufficient isolation. An attacker can embed malicious payloads into these external channels to hijack the agent's planning and execution trace. This allows the attacker to silently trigger high-privilege actions via OpenClaw's Skills…