Language Model Security Database

A novel attack, dubbed PRP (Propagating Universal Perturbations), bypasses guardrail LLMs by constructing a universal adversarial prefix that, when prepended to any harmful response, evades detection by the guard model. This prefix is then propagated to the base LLM's response using in-context learning, causing the guardrail LLM to generate harmful content.

Large Language Models (LLMs) used for zero-shot text assessment are vulnerable to universal adversarial attacks. Concatenating short phrases ("universal adversarial phrases") to assessed text can artificially inflate the predicted scores, regardless of the actual quality of the text. This vulnerability is particularly pronounced in LLMs performing absolute scoring, as opposed to comparative assessment.

Large language models (LLMs) are vulnerable to jailbreaking attacks that exploit human-like persuasive techniques rather than algorithmic or technical flaws. Attackers can craft prompts ("Persuasive Adversarial Prompts" or PAPs) leveraging social influence strategies (e.g., logical appeal, emotional appeal, authority endorsement) to elicit responses that violate safety guidelines and reveal sensitive or harmful information. The effectiveness of these attacks surpasses traditional…

A vulnerability in the safety alignment of large language models (LLMs) allows a "weak-to-strong" jailbreaking attack. This attack uses a smaller, adversarially trained ("unsafe") LLM to manipulate the decoding probabilities of a much larger, safety-aligned ("safe") LLM, leading the larger model to generate harmful outputs. The attack leverages the observation that the initial decoding distributions of safe and unsafe LLMs differ significantly, but this difference diminishes as the generation…

A vulnerability exists in large language models (LLMs) allowing for the injection of persistent backdoors via fine-tuning with a crafted dataset. The backdoor triggers the LLM to generate unsafe outputs for specific harmful prompts, while remaining undetected during standard safety audits due to the trigger's design and the backdoor's persistence against re-alignment techniques. The attack leverages elongated triggers, unlike previous attacks which used shorter triggers easily removed via…

Large Language Models (LLMs) such as Llama 2 and Vicuna exhibit a vulnerability where specific layers (e.g., layer 3 in Llama2-13B, layer 1 in Llama2-7B and Vicuna-13B) overfit to harmful prompts, resulting in a disproportionate influence on the model's output for such prompts. This overfitting creates a narrow "safety" mechanism easily bypassed by adversarial prompts designed to avoid triggering these specific layers. Additionally, a single neuron (e.g., neuron 2100 in Llama2 and Vicuna)…

A vulnerability in Text-to-Image (T2I) models' safety filters allows bypassing through the injection of adversarial prompts crafted by an LLM-driven multi-agent system. The attack, named Divide-and-Conquer Attack (DACA), circumvents the filters by rephrasing harmful prompts into multiple benign descriptions of individual visual components, thus avoiding detection while maintaining the original visual intent.

Large Language Models (LLMs) with accessible output logits are vulnerable to "coercive interrogation," a novel attack that extracts harmful knowledge hidden in low-ranked tokens. The attack doesn't require crafted prompts; instead, it iteratively forces the LLM to select and output low-probability tokens at key positions in the response sequence, revealing toxic content the model would otherwise suppress.

Large Language Models (LLMs) exhibit an inherent response tendency, predisposing them towards affirmation or rejection of instructions. The RADIAL attack exploits this tendency by strategically inserting real-world instructions, identified as inherently inducing affirmation responses, around malicious prompts. This bypasses LLM safety mechanisms, resulting in the generation of harmful content.

A vulnerability exists in large language models (LLMs) utilizing in-context learning (ICL). Malicious actors can inject imperceptible adversarial suffixes into in-context demonstrations, causing the LLM to generate targeted, unintended outputs, even when the user query is benign. The attack manipulates the LLM's attention mechanism, diverting it towards the adversarial tokens.