Automated LLM Jailbreak

Description

Large Language Models (LLMs) are vulnerable to prompt-based jailbreaks, allowing adversaries to bypass safety guardrails and elicit undesirable outputs. The Prompt Automatic Iterative Refinement (PAIR) algorithm efficiently generates these jailbreaks using a limited number of black-box queries to the target LLM. The vulnerability stems from the LLM's inability to robustly handle adversarial prompts crafted through iterative refinement, even without white-box access to its internal mechanisms.

Examples

See the accompanying paper for specific examples of prompt-level jailbreaks generated by PAIR against GPT-3.5 Turbo, GPT-4, Vicuna, Gemini Pro, and other evaluated models. Examples include prompts designed to elicit instructions for building bombs, creating phishing emails, and generating hate speech. The paper provides detailed examples of prompt and response pairs that successfully jailbreak the target LLMs.

Impact

Successful exploitation allows adversaries to circumvent the safety mechanisms of LLMs, leading to the generation of harmful, illegal, biased, or otherwise undesirable content. This can be used for purposes including disinformation campaigns, hate speech generation, and the creation of illegal instructions.

Affected Systems

LLMs susceptible to prompt-based jailbreaks, including the evaluated GPT-3.5 Turbo, GPT-4, Vicuna 13B v1.5, Gemini Pro, Llama 2 7B Chat, Claude Instant 1.2, Claude 2.1, and Mixtral 8x7B Instruct models.

Mitigation Steps

• Improved Prompt Filtering: Implement more sophisticated prompt filtering techniques that can identify and block adversarial prompts, including those generated through iterative refinement strategies. This might involve incorporating techniques designed to detect semantic similarities to known adversarial prompts.
• Reinforcement Learning from Human Feedback (RLHF) Improvements: Enhance RLHF training processes to better handle adversarial scenarios and improve the model's resilience against manipulative prompts. This may involve training on a wider range of adversarial prompts.
• Defense Models: Integrate defense models that identify and neutralize adversarial prompts before they reach the core LLM. This could involve using additional LLMs to evaluate the potential harm of an input prompt before processing.
• Regular Red Teaming: Conduct regular red teaming exercises using techniques like PAIR to proactively identify and address vulnerabilities in LLM safety mechanisms.