Language Model Security Database

Large Language Models (LLMs) are vulnerable to prompt injection attacks that can bypass their internal copyright compliance mechanisms, causing them to generate verbatim copyrighted text. The vulnerability stems from insufficient robustness against prompt engineering techniques that manipulate the model into ignoring or circumventing its safety filters designed for copyright protection.

Large Language Models (LLMs) used to control robots exhibit biases leading to discriminatory and unsafe behaviors. When provided with personal characteristics (e.g., race, gender, disability), LLMs generate biased outputs resulting in discriminatory actions (e.g., assigning lower rescue priority to certain groups) and accept or deem feasible dangerous or unlawful instructions (e.g., removing a person's mobility aid).

LLMs, even when individually assessed as "safe," can be combined by an adversary to achieve malicious outcomes. This vulnerability exploits the complementary strengths of multiple models—a high-capability model that refuses malicious requests and a low-capability model that does not—through task decomposition. Adversaries can either manually decompose tasks into benign (solved by the high-capability model) and easily-malicious subtasks (solved by the low-capability model) or automate the…

Large language models (LLMs) are vulnerable to jailbreaking attacks using adversarially generated suffixes. The AmpleGCG attack generates a large number of diverse, effective suffixes which bypass safety mechanisms in both open and closed-source LLMs. The attack leverages the observation that low loss during suffix generation is not a reliable indicator of jailbreaking success, and generates diverse suffixes from intermediate steps of the optimization process.

A color-aware attack, Self Color Testing-based Substitution (SCTS), bypasses watermarking mechanisms in LLMs designed to identify AI-generated text. SCTS exploits the LLM's compliance with instructions to infer the "color" (green/red token classification) of tokens, allowing for targeted substitution of watermarked tokens with non-watermarked tokens, thus evading watermark detection. The attack is particularly effective against watermarks that utilize logit perturbation to bias token selection.

Large Language Models (LLMs) are vulnerable to a novel prompting technique, "conditional Variational-autoencoder-Like Prompt" (VLPrompt), which enables the generation of highly convincing fake news articles. VLPrompt overcomes limitations of previous methods by eliminating the need for additional human-collected data while maintaining contextual coherence and detail. This allows for the automated mass-production of realistic-sounding fake news.

Large Language Models (LLMs) such as Llama 2 and Vicuna exhibit a vulnerability where specific layers (e.g., layer 3 in Llama2-13B, layer 1 in Llama2-7B and Vicuna-13B) overfit to harmful prompts, resulting in a disproportionate influence on the model's output for such prompts. This overfitting creates a narrow "safety" mechanism easily bypassed by adversarial prompts designed to avoid triggering these specific layers. Additionally, a single neuron (e.g., neuron 2100 in Llama2 and Vicuna)…

AutoDAN is an interpretable gradient-based adversarial attack that generates readable prompts to bypass perplexity filters and jailbreak LLMs. The attack crafts prompts that elicit harmful behaviors while maintaining sufficient readability to avoid detection by existing perplexity-based defenses. This is achieved through a left-to-right token-by-token generation process optimizing for both jailbreaking success and prompt readability.

Large Language Models (LLMs) employing Reinforcement Learning from Human Feedback (RLHF) and instruction tuning methods may exhibit superficial safety guardrails vulnerable to parametric red-teaming attacks. Fine-tuning the model on a dataset of harmful prompts and their corresponding helpful (but harmful) responses can bypass built-in safety mechanisms, resulting in the model generating unsafe outputs. This vulnerability is demonstrated by achieving an 88% success rate in eliciting harmful…