RoleBreak Character Jailbreak
Large Language Models (LLMs) used in role-playing systems are vulnerable to character hallucination attacks, a form of jailbreak exploiting "query sparsity" and "role-query conflict". Query sparsity occurs when prompts fall outside the model's training data distribution, causing it to generate out-of-character responses. Role-query conflict arises when the prompt contradicts the established character persona, leading to inconsistent behavior. These vulnerabilities allow attackers to elicit…
Evaluated models: Claude 3 Haiku, GPT-3.5 Turbo, Llama 3 8B+1 more
- Claude 3 Haiku
- GPT-3.5 Turbo
- Llama 3 8B
- Mistral 7B Instruct v0.2