Kov: MDP-Based LLM Jailbreak

Description

Large Language Models (LLMs) are vulnerable to naturalistic adversarial attacks crafted using Markov Decision Processes (MDPs) and Monte Carlo Tree Search (MCTS). These attacks generate natural-language prompts that elicit harmful, violent, or discriminatory responses from the LLMs, even those with built-in safety mechanisms. The attacks are transferable across different LLMs, demonstrating a generalized vulnerability.

Examples

See arXiv:2408.08899 (opens in a new tab) for the paper's evaluated prompts and responses. The paper demonstrates that a harmful-request template followed by a short, naturalistic adversarial suffix can elicit a prohibited response from GPT-3.5; the operational harmful prompt is intentionally omitted here.

Impact

Successful exploitation of this vulnerability leads to the generation of harmful content by the LLM, potentially resulting in:

• Dissemination of harmful information: LLMs can be tricked into providing instructions for illegal activities, hate speech, or violence.
• Erosion of trust: The vulnerability undermines confidence in the safety and reliability of LLMs.
• Reputational damage: Organizations deploying vulnerable LLMs can suffer reputational harm.

Affected Systems

The vulnerability affects various LLMs, including but not limited to GPT-3.5 and other models susceptible to token-level adversarial attacks. Newer models like GPT-4 may exhibit increased resistance, but the vulnerability's transferability suggests potential impact on future models.

Mitigation Steps

• Enhance LLM training data with more diverse and robust examples of adversarial prompts to improve model robustness against attacks.
• Develop and implement more sophisticated safety filters and monitoring systems capable of detecting and mitigating natural language adversarial attacks.
• Implement techniques to identify and reject prompts detected as potentially adversarial.
• Introduce additional layers of security and content moderation to catch harmful outputs. These should go beyond simple keyword filters and incorporate more nuanced analysis to catch subtly phrased harmful requests.