Language Model Security Database

Large Language Models (LLMs), specifically open-weight instruction-tuned models (including Llama-3.1-8B, Qwen3-8B, and Gemma-7B) and certain closed-weight APIs allowing partial response pre-filling, are vulnerable to "Sockpuppetting" or Output Prefix Injection. This vulnerability exploits the model's autoregressive nature and self-consistency bias. By injecting a target acceptance sequence (e.g., "Sure, here is...") directly into the start of the assistant message block within the chat…

A vulnerability exists in Large Language Model (LLM) Fine-tuning-as-a-Service (FaaS) platforms that allows attackers to bypass safety alignment and moderation filters via a "TrojanPraise" benign fine-tuning attack. The attack exploits the decoupling of an LLM's internal representation of harmful queries into "knowledge" (semantic understanding) and "attitude" (safety refusal). The attacker constructs a fine-tuning dataset containing three specific components: (1) a novel nonsense word (e.g…

A vulnerability exists in frontier Large Language Models (LLMs) where in-context information (e.g., provided via Retrieval-Augmented Generation) completely overrides parametric safety guardrails when processing counterfactual or adversarial medical evidence. When a prompt contains fabricated clinical context asserting the medical efficacy of toxic substances, illicit drugs, or nonsensical items, the LLM suppresses its internal knowledge of the substance's toxicity. Internal representation…

Large Reasoning Models (LRMs) employing Chain-of-Thought (CoT) generation are vulnerable to sensitive information leakage through intermediate reasoning steps, even after undergoing standard unlearning procedures (such as Gradient Ascent, Direct Preference Optimization, or KL Minimization). While these fine-tuning-based unlearning methods typically suppress sensitive content in the final generated answer, they fail to purge the information from the model's internal reasoning trajectory…

A vulnerability exists in the function-calling mechanisms of open-source Large Language Models (LLMs), specifically identified as "Renaming Tool Poisoning" (RTP). This attack vector exploits the model's visibility into both the natural language description and the actual code implementation of available tools. By embedding a two-part adversarial payload—one in the tool description directing focus to implementation variables, and another within the tool's source code variable…

A vulnerability exists in the task-planning and execution logic of Large Language Model (LLM) agents, specifically within trip-planning and web-use agents. The vulnerability, identified as a "User-Mediated Attack," occurs because agents prioritize task completion and "helpfulness" over safety verification when processing content provided by the user. When a benign user forwards untrusted external content (e.g., promotional text containing phishing links or malicious instructions) to the agent…

Large Language Models (LLMs) and their associated output guardrails (e.g., Llama Guard, OpenAI Moderation API) rely on autoregressive, token-by-token processing, which interprets text as a one-dimensional sequence. A vulnerability exists wherein harmful content can bypass these safety filters by exploiting the discrepancy between 1D token serialization and 2D visual rendering. By redistributing tokens across different rows, columns, or diagonals (SpatialJB), attackers can induce the model to…

Large Language Models (LLMs) exhibit a False Refusal vulnerability during legitimate hate speech detoxification tasks (text style transfer). Safety alignment mechanisms fail to contextually distinguish between a benign instruction to "detoxify" or "rewrite" harmful content and the generation of harmful content itself. This results in a denial of service where the model refuses to process the input. This vulnerability is not uniformly distributed; it is statistically biased to…

Reasoning-capable Large Language Models (LLMs) and agentic AI systems exhibit a critical vulnerability to contextual distractors, resulting in catastrophic performance degradation (up to 80% drop in accuracy) and emergent misalignment. When the input context contains noise—specifically random documents, irrelevant chat history, or task-specific "hard negative" distractors—the models fail to filter this information. Instead of ignoring the noise, the models disproportionately attend to…

Text-to-Image (T2I) models and their associated safety filters are vulnerable to MacPrompt, a black-box jailbreak technique that exploits cross-lingual embedding alignments. Attackers can bypass input text filters, latent representation filters, and model-level concept removal defenses by replacing sensitive keywords with "macaronic" substitutes. These substitutes are constructed by extracting and recombining character-level substrings from translations of the target word across multiple…