Language Model Security Database

Large Language Models (LLMs) employing safety mechanisms based on supervised fine-tuning and preference alignment exhibit a vulnerability to "steering" attacks. Maliciously crafted prompts or input manipulations can exploit representation vectors within the model to either bypass censorship ("refusal-compliance vector") or suppress the model's reasoning process ("thought suppression vector"), resulting in the generation of unintended or harmful outputs. This vulnerability is demonstrated…

Large Language Models (LLMs) are vulnerable to attacks that generate obfuscated activations, bypassing latent-space defenses such as sparse autoencoders, representation probing, and latent out-of-distribution (OOD) detection. Attackers can manipulate model inputs or training data to produce outputs exhibiting malicious behavior while remaining undetected by these defenses. This occurs because the models can represent harmful behavior through diverse activation patterns, allowing attackers to…

A vulnerability exists in large language models (LLMs) where targeted bitwise corruptions in model parameters can induce a "jailbroken" state, causing the model to generate harmful responses without input modification. Fewer than 25 bit-flips are sufficient to achieve this in many cases. The vulnerability stems from the susceptibility of the model's memory representation to fault injection attacks.

Large Language Models (LLMs) are vulnerable to efficient adversarial attacks using Projected Gradient Descent (PGD) on a continuously relaxed input prompt. This attack bypasses existing alignment methods by crafting adversarial prompts that induce the model to produce undesired or harmful outputs, significantly faster than previous state-of-the-art discrete optimization methods. The effectiveness stems from carefully controlling the error introduced by the continuous relaxation of the discrete…

Large Language Models (LLMs) such as Llama 2 and Vicuna exhibit a vulnerability where specific layers (e.g., layer 3 in Llama2-13B, layer 1 in Llama2-7B and Vicuna-13B) overfit to harmful prompts, resulting in a disproportionate influence on the model's output for such prompts. This overfitting creates a narrow "safety" mechanism easily bypassed by adversarial prompts designed to avoid triggering these specific layers. Additionally, a single neuron (e.g., neuron 2100 in Llama2 and Vicuna)…