Ensemble Black-box Jailbreak

Description

Large Language Models (LLMs) are vulnerable to transferable ensemble black-box jailbreak attacks. The vulnerability allows an attacker to bypass safety mechanisms and elicit undesired or harmful responses from the LLM by using an ensemble of LLM-as-attacker methods that optimize malicious prompts, adaptively adjusting resources based on prompt difficulty, and strategically modifying prompt semantics to evade detection.

Examples

See arXiv:2410.23558 (opens in a new tab) for concrete examples from the paper.

Impact

Successful exploitation could lead to LLMs generating responses that violate their intended safety constraints, providing harmful instructions, biased information, or revealing sensitive data. The transferability of the attack allows its use across multiple LLMs.

Affected Systems

Multiple large language models (LLMs). Specific models affected are not explicitly listed in the research but include Gemma-2B-IT, Gemma2-9B-IT (targets) and Llama3-8B-Instruct, GLM-4-Plus, GLM-4-Flash, Qwen-Max-Latest, and DeepSeek-V2.5 (judges).

Mitigation Steps

• Implement robust prompt filtering mechanisms that go beyond simple keyword matching.
• Develop defense strategies that are resistant to ensemble attacks.
• Improve or diversify the internal embedding representations within the LLMs to make them less vulnerable to semantic manipulation.
• Employ more sophisticated mechanisms for detecting and mitigating adversarial prompt crafting techniques.
• Regularly audit and update the LLMs’ safety mechanisms against new attack vectors.