Language Model Security Database

Large Reasoning Models (LRMs) employing Chain-of-Thought (CoT) generation are vulnerable to sensitive information leakage through intermediate reasoning steps, even after undergoing standard unlearning procedures (such as Gradient Ascent, Direct Preference Optimization, or KL Minimization). While these fine-tuning-based unlearning methods typically suppress sensitive content in the final generated answer, they fail to purge the information from the model's internal reasoning trajectory…

Unintended input-only PII memorization in fine-tuned Large Language Models (LLMs) allows remote attackers to extract sensitive Personally Identifiable Information (PII) such as names, medical records, and financial details. This vulnerability occurs when a model is fine-tuned on datasets where sensitive information appears in the input text, even if that information is not part of the training target (label) or is unrelated to the downstream task (e.g., classification). The fine-tuning process…

The Model Context Protocol (MCP) specification v1.0 contains fundamental architectural vulnerabilities enabling server-side prompt injection and privilege escalation. The protocol relies on bidirectional sampling (sampling/createMessage) without cryptographic origin authentication or UI distinction, allowing connected servers to inject content that the LLM backend interprets as legitimate user input. Additionally, the protocol lacks isolation boundaries between concurrent server connections…

Production Large Language Models (LLMs) are vulnerable to long-form training data extraction via a two-phase prompt injection attack. This vulnerability allows an attacker to recover substantial portions of memorized, copyrighted text (such as novels) by exploiting the model's autoregressive text completion capabilities. The attack methodology involves two distinct phases: 1. Prefix Completion Probe: The attacker provides a short "seed" sequence (e.g., the first sentence of a book) coupled…

Large Language Model (LLM) agents operating in tool-augmented environments are susceptible to "Contextual Fragility" and multi-turn "long-chain" exploitation. Existing safety mechanisms predominantly function on a stateless, atomic paradigm, evaluating individual input-output pairs in isolation. This allows an adversary to orchestrate complex attack trajectories where malicious intent is distributed across multiple, individually benign steps (a "Domino Effect"). Consequently, an attacker can…

A black-box guardrail reverse-engineering vulnerability exists in Large Language Model (LLM) serving systems that employ output filtering mechanisms. The vulnerability allows remote attackers to replicate the proprietary decision-making policy and rule sets of the target's safety guardrail without direct access to model parameters. This is achieved through a technique termed Guardrail Reverse-engineering Attack (GRA), which utilizes a reinforcement learning framework combined with genetic…

A vulnerability exists in certain Large Language Models and diffusion models due to discontinuities in their latent space, which arise from data sparsity during training. An attacker can craft inputs containing lexically rare or semantically ambiguous constructs to guide the model's inference process toward these unstable, poorly-conditioned regions. This technique, termed "Alignment Degradation Induction," can degrade or bypass safety alignment mechanisms. Through iterative, multi-turn…

Multiple open-weight Large Language Models (LLMs)—specifically those prioritizing capability over safety alignment—exhibit a critical vulnerability to adaptive multi-turn prompt injection and jailbreak attacks. While these models effectively reject isolated, single-turn adversarial inputs (averaging ~13.11% Attack Success Rate), they fail to maintain safety guardrails and policy enforcement across extended conversational contexts. By leveraging iterative strategies such as "Crescendo" (gradual…

A vulnerability termed "Controlled-Release Prompting" allows attackers to bypass lightweight input filters (prompt guards) deployed in front of Large Language Models (LLMs). The attack exploits the computational resource asymmetry between the resource-constrained guard model and the highly capable target model. Attackers encode malicious instructions using obfuscation techniques—such as substitution ciphers (Timed-Release) or verbose character descriptions (Spaced-Release)—that require…

A distributed backdoor vulnerability, named "Collaborative Shadows", exists in LLM-based Multi-Agent Systems (MAS) that rely on external or modifiable tools. An attacker can poison multiple agent tools by embedding inert, encrypted "attack primitives" within them. These primitives are fragments of a larger malicious payload. A carefully crafted user instruction acts as both a trigger and a decryption key. The instruction steers the agents to collaborate in a specific sequence, causing them to…