Language Model Security Database

Low-rank adaptation (LoRA) fine-tuning allows efficient circumvention of safety training in large language models (LLMs), such as Llama 2-Chat 70B, resulting in significantly reduced refusal rates for harmful prompts while maintaining general performance capabilities. Attackers can use LoRA with a small, synthetic dataset of harmful instructions and responses to effectively undo safety measures implemented during the model's training.

A vulnerability exists in large language models (LLMs) allowing for the injection of persistent backdoors via fine-tuning with a crafted dataset. The backdoor triggers the LLM to generate unsafe outputs for specific harmful prompts, while remaining undetected during standard safety audits due to the trigger's design and the backdoor's persistence against re-alignment techniques. The attack leverages elongated triggers, unlike previous attacks which used shorter triggers easily removed via…

Large Language Models (LLMs), specifically Llama 3 8B and 70B, are vulnerable to a rapid removal of safety fine-tuning through parameter-efficient fine-tuning (PEFT) methods. Attackers with access to model weights can use techniques like QLoRA, ReLoRA, or Ortho to effectively circumvent safety mechanisms in a matter of minutes using readily available computational resources. This allows bypassing safety restrictions and eliciting unsafe outputs.

A vulnerability in Text-to-Image (T2I) models' safety filters allows bypassing through the injection of adversarial prompts crafted by an LLM-driven multi-agent system. The attack, named Divide-and-Conquer Attack (DACA), circumvents the filters by rephrasing harmful prompts into multiple benign descriptions of individual visual components, thus avoiding detection while maintaining the original visual intent.

Large Language Models (LLMs) used for code generation are vulnerable to adversarial natural language instructions that preserve semantic meaning but induce the generation of functionally correct code containing specific vulnerabilities. The attack leverages a novel algorithm, DeceptPrompt, to generate adversarial prompts that manipulate the LLM's output, resulting in vulnerable code without altering the intended functionality.

Fine-tuning aligned Large Language Models (LLMs) on a small number of adversarially crafted examples, or even on benign datasets, can compromise their safety alignment, leading to the generation of harmful or inappropriate content. This vulnerability exploits the few-shot learning capabilities of LLMs, allowing attackers to override existing safety mechanisms with minimal effort and cost. Even unintentional fine-tuning with seemingly benign datasets can result in unintended safety degradation.

Large Language Models (LLMs) such as Llama 2 and Vicuna exhibit a vulnerability where specific layers (e.g., layer 3 in Llama2-13B, layer 1 in Llama2-7B and Vicuna-13B) overfit to harmful prompts, resulting in a disproportionate influence on the model's output for such prompts. This overfitting creates a narrow "safety" mechanism easily bypassed by adversarial prompts designed to avoid triggering these specific layers. Additionally, a single neuron (e.g., neuron 2100 in Llama2 and Vicuna)…

Large Language Models (LLMs) are vulnerable to Compositional Instruction Attacks (CIA), where malicious prompts are embedded within seemingly harmless instructions. This allows attackers to bypass safety mechanisms and elicit harmful responses from the model, even if the individual components of the prompt would be flagged as safe. The attack exploits the model's inability to correctly identify underlying malicious intent within composite instructions.

Large Language Models (LLMs) are vulnerable to adversarial prompts generated by the Rainbow Teaming technique. Rainbow Teaming uses a quality-diversity search algorithm to create a diverse set of prompts that elicit unsafe, biased, or incorrect outputs from the target LLM, exceeding a 90% success rate across various models. The vulnerability stems from the LLMs' susceptibility to these carefully crafted prompts, bypassing existing safety mechanisms. These prompts are highly transferable across…

Large Language Models (LLMs) employing Reinforcement Learning from Human Feedback (RLHF) and instruction tuning methods may exhibit superficial safety guardrails vulnerable to parametric red-teaming attacks. Fine-tuning the model on a dataset of harmful prompts and their corresponding helpful (but harmful) responses can bypass built-in safety mechanisms, resulting in the model generating unsafe outputs. This vulnerability is demonstrated by achieving an 88% success rate in eliciting harmful…