Iterative Chaos Jailbreak

Description

Large Reasoning Models (LRMs) are vulnerable to a novel jailbreak attack, "Mousetrap," which leverages the models' reasoning capabilities to elicit harmful responses. Mousetrap uses a "Chaos Machine" to iteratively transform prompts via one-to-one mappings (e.g., character substitutions, word reversals), creating complex reasoning chains that confuse the LRM and cause it to generate unsafe outputs despite safety mechanisms. The iterative nature of the attack, combined with role-playing prompts, increases the likelihood of bypassing safety filters.

Examples

See the paper's Appendix G for examples of successful Mousetrap attacks against various LRMs, including prompts requesting detailed instructions for illegal activities. These examples demonstrate the generation of harmful content that surpasses the capabilities of previous attacks on LLMs.

Impact

Successful exploitation of this vulnerability can lead to the generation of detailed instructions for illegal activities, hate speech, and other harmful content, potentially causing significant harm to individuals and society. The attack is shown to be highly effective even against LRMs designed with enhanced safety features.

Affected Systems

The vulnerability affects various Large Reasoning Models, including but not limited to OpenAI's o1-mini, Anthropic's Claude-sonnet, and Google's Gemini-thinking. The paper indicates that the attack's effectiveness is linked to the strength of the model's reasoning capabilities.

Mitigation Steps

• Improve LRM safety mechanisms to detect and mitigate the iterative prompt transformations used in the Mousetrap attack.
• Develop more robust defenses against role-playing prompts and other techniques that attempt to manipulate the model's reasoning process and context.
• Enhance the model's ability to identify and reject toxic prompts even after undergoing multiple transformations.
• Implement more rigorous output filtering to detect and remove harmful content generated in response to seemingly innocuous or disguised prompts.