Happy Ending LLM Jailbreak

Description

Large language models (LLMs) exhibit increased responsiveness to prompts framed within positive narratives. The Happy Ending Attack (HEA) exploits this by embedding malicious requests within a positive-sentiment scenario culminating in a happy ending. This allows the LLM to generate responses that fulfill the malicious request while perceiving the overall prompt as benign.

Examples

See the paper "Dagger Behind Smile: Fool LLMs with a Happy Ending Story" for examples demonstrating successful HEA attacks across multiple LLMs (GPT-4, Gemini, Llama). Examples include generating instructions for creating explosive devices, committing insider trading, and crafting disinformation campaigns, all embedded in stories with positive outcomes.

Impact

Successful HEA attacks can lead to the generation of unsafe, illegal, or unethical content by LLMs, compromising their safety and reliability. The attack is highly effective against various LLMs, including state-of-the-art models, bypassing existing safety mechanisms. The relative simplicity of the attack also facilitates its widespread implementation.

Affected Systems

All LLMs vulnerable to prompt injection attacks are potentially affected. This includes, but is not limited to, GPT-4, Gemini, and Llama models. The paper demonstrated the attack's effectiveness across a range of model sizes from the same family.

Mitigation Steps

• Improve LLM safety mechanisms by enhancing their ability to detect malicious intent even within positive narratives.
• Develop techniques to better assess the overall sentiment and intent of a prompt, going beyond simple keyword analysis.
• Implement more robust defenses against prompt injection attacks, adapting to the evolving nature of such attacks.
• Develop more sophisticated detection mechanisms to identify patterns indicative of HEA-style attacks.
• Regularly audit LLMs for vulnerabilities to this type of attack and implement necessary patches.