Language Model Security Database

Large Language Models (LLMs) exhibit Defense Threshold Decay (DTD): generating substantial benign content shifts the model's attention from the input prompt to prior outputs, increasing susceptibility to jailbreak attacks. The "Sugar-Coated Poison" (SCP) attack exploits this by first generating benign content, then transitioning to malicious output.

A vulnerability exists in Large Language Model (LLM) decision-making capabilities described as "Rhetorical Persuasion Override." When an LLM is deployed as a judge or evaluator in a single-turn, multi-agent debate framework, it systematically fails to distinguish factual truth from confidently presented misinformation. An adversarial agent can coerce the evaluator into endorsing a known falsehood from the TruthfulQA dataset by employing specific rhetorical strategies—namely, high confidence…

A vulnerability exists in the combination of Large Language Models (LLMs) and their associated safety guardrails, allowing attackers to bypass both defenses and elicit harmful or unintended outputs from LLMs. The vulnerability stems from insufficient detection by guardrails against adversarially crafted prompts, which appear benign but contain hidden malicious intent. The attack, dubbed "DualBreach," leverages a target-driven initialization strategy and multi-target optimization to generate…

State-of-the-art Reward Models (RMs) utilized in Reinforcement Learning from Human Feedback (RLHF) exhibit poor out-of-distribution (OOD) generalization, making them susceptible to adversarial inputs. These models fail to reliably assess prompt-response pairs that diverge from their training distribution, assigning high reward scores to low-quality, nonsensical, or syntactically incorrect responses. This vulnerability allows for "reward hacking," where a policy model optimizes for unintended…

A vulnerability in Large Language Models (LLMs) allows attackers to bypass safety mechanisms and elicit detailed harmful responses by strategically manipulating input prompts. The vulnerability exploits the LLM's sensitivity to "scenario shifts"—contextual changes in the input that influence the model's output, even when the core malicious request remains the same. A genetic algorithm can optimize these scenario shifts, increasing the likelihood of obtaining detailed harmful responses while…

Large Language Models (LLMs) employing alignment safeguards and safety mechanisms are vulnerable to graph-based adversarial attacks that bypass these protections. The attack, termed "Graph of Attacks" (GOAT), leverages a graph-based reasoning framework to iteratively refine prompts and exploit vulnerabilities more effectively than previous methods. The attack synthesizes information across multiple reasoning paths to generate human-interpretable prompts that elicit undesired or harmful outputs…

Improper Input Validation in Large Language Model (LLM) systems configured as automated evaluators ("LLM-as-a-judge") allows remote attackers to manipulate evaluation scores and comparative verdicts via adversarial prompt injection. The vulnerability arises when the model processes untrusted input containing linguistic masquerading, context separators, and disruptor commands (e.g., "Basic Injection", "Contextual Misdirection", and "Adaptive Search-Based Attack"). Successful exploitation…

Large Language Models (LLMs) employing safety mechanisms based on supervised fine-tuning and preference alignment exhibit a vulnerability to "steering" attacks. Maliciously crafted prompts or input manipulations can exploit representation vectors within the model to either bypass censorship ("refusal-compliance vector") or suppress the model's reasoning process ("thought suppression vector"), resulting in the generation of unintended or harmful outputs. This vulnerability is demonstrated…

Large Language Model (LLM) guardrail systems, including those relying on AI-driven text classification models (e.g., fine-tuned BERT models), are vulnerable to evasion via character injection and adversarial machine learning (AML) techniques. Attackers can bypass detection by injecting Unicode characters (e.g., zero-width characters, homoglyphs) or using AML to subtly perturb prompts, maintaining semantic meaning while evading classification. This allows malicious prompts and jailbreaks to…

Multi-Agent Debate (MAD) frameworks leveraging Large Language Models (LLMs) are vulnerable to amplified jailbreak attacks. A novel structured prompt-rewriting technique exploits the iterative dialogue and role-playing dynamics of MAD, circumventing inherent safety mechanisms and significantly increasing the likelihood of generating harmful content. The attack succeeds by using narrative encapsulation, role-driven escalation, iterative refinement, and rhetorical obfuscation to guide agents…