Exponentiated Gradient Jailbreak
A vulnerability in several open-source Large Language Models (LLMs) allows attackers using exponentiated gradient descent to craft adversarial prompts that cause the models to generate harmful or unintended outputs, effectively "jailbreaking" the safety alignment mechanisms. The attack optimizes a continuous relaxed one-hot encoding of the input tokens, intrinsically satisfying constraints, and avoiding the need for projection techniques used in previous methods.
Evaluated models: Beaver 7B v1.0 Cost, Falcon 7B Instruct, Llama 2 7B Chat+4 more
- Beaver 7B v1.0 Cost
- Falcon 7B Instruct
- Llama 2 7B Chat
- Llama 3 8B Instruct
- Mistral 7B v0.3
- MPT 7B Chat
- Vicuna 7B v1.5