Carrier Article Jailbreak

Description

Large Language Models (LLMs) are vulnerable to a novel jailbreak attack that leverages "neural carrier articles." This attack injects a prohibited query into a benign article generated by a secondary LLM, designed to be semantically similar to the prohibited query but not trigger the primary LLM's safety mechanisms. The secondary LLM generates articles based on hypernyms derived from the prohibited query, thus subtly shifting attention weights within the primary LLM, bypassing its safeguards.

Examples

The paper provides examples of successful exploits against several LLMs (Llama-2-7B, Llama-3-8B, Gemini, GPT-3.5-Turbo, GPT-4) using various prohibited queries (e.g., "how to produce dynamite," "how to insult the president"). Specific prompts and responses are presented in arXiv:2408.11182 (opens in a new tab).

Impact

Successful exploitation allows adversaries to bypass safety restrictions imposed on LLMs, leading to the generation of malicious, unethical, or illegal content. This compromises the intended safety and security of the affected LLMs and applications using them.

Affected Systems

The vulnerability affects various LLMs including, but not limited to, Llama-2 7B, Llama-3-8b, Gemini, GPT-3.5-turbo, GPT-4. The attack's success is LLM-specific and depends on the specific safety mechanisms implemented.

Mitigation Steps

• Improve LLM safety mechanisms to be more robust against subtle semantic manipulations.
• Implement more sophisticated prompt filtering and analysis techniques that go beyond keyword matching.
• Develop LLMs with improved context and attention management capabilities to prevent exploitation by strategically inserted text.
• Consider incorporating anomaly detection mechanisms to identify and flag suspicious prompts based on their context and semantic similarity to known prohibited queries.