Language Model Security Database

Fine-tuning aligned Large Language Models (LLMs) on a small number of adversarially crafted examples, or even on benign datasets, can compromise their safety alignment, leading to the generation of harmful or inappropriate content. This vulnerability exploits the few-shot learning capabilities of LLMs, allowing attackers to override existing safety mechanisms with minimal effort and cost. Even unintentional fine-tuning with seemingly benign datasets can result in unintended safety degradation.

Low-rank adaptation (LoRA) fine-tuning allows efficient circumvention of safety training in large language models (LLMs), such as Llama 2-Chat 70B, resulting in significantly reduced refusal rates for harmful prompts while maintaining general performance capabilities. Attackers can use LoRA with a small, synthetic dataset of harmful instructions and responses to effectively undo safety measures implemented during the model's training.

Large Language Models (LLMs) are vulnerable to a "Chain of Utterances" (CoU) based prompt injection attack. This attack exploits the LLM's ability to engage in multi-turn conversations and role-playing, tricking it into providing harmful or unsafe responses even when presented with safety guidelines. The attack leverages a crafted conversation between two agents ("Red-LM," a malicious agent, and "Base-LM," a seemingly helpful agent) to elicit unethical responses from the Base-LM by subtly…