Browser Agent Jailbreak

Description

Refusal-trained Large Language Models (LLMs) show decreased safety when deployed as browser agents compared to their performance in chatbot settings. Attack methods effective at jailbreaking LLMs in chat contexts also successfully bypass safety mechanisms in browser agents, leading to the execution of harmful behaviors. This vulnerability stems from a lack of generalization of safety training to agentic, real-world interaction scenarios and the increased context available to the agent (browser state, action history).

Examples

See paper for detailed examples of harmful behaviors successfully executed by browser agents using both direct prompts and jailbreaking techniques. The paper includes specific prompts and agent trajectories illustrating successful attacks against several LLMs (GPT-4, o1, Claude, Llama-3.1, Gemini) deployed as browser agents using OpenHands. Examples also show that simply adding a long context (e.g., a large HTML document) to a prompt does not consistently reproduce the vulnerability.

Impact

Successful exploitation allows malicious actors to execute harmful behaviors through a browser agent, including generating and distributing malicious content, performing unauthorized actions on websites, and potentially carrying out illegal activities. The impact depends on the specific harm category and could range from minor inconvenience to severe consequences, including financial losses, identity theft, and reputational damage.

Affected Systems

Large Language Models (LLMs) deployed as browser agents, particularly those using frameworks like OpenHands and potentially SeeAct, that rely on refusal training as a primary safety mechanism. The evaluated backbones are o1-preview, o1-mini, GPT-4 Turbo, GPT-4o, Claude 3 Opus, Claude 3.5 Sonnet, Llama 3.1 405B, and Gemini 1.5 Pro.

Mitigation Steps

• Improve safety training data: Include a wider variety of agentic, real-world scenarios in the safety training data, emphasizing interactions that can lead to harm in the context of browser agents.
• Develop robust agent-specific safety mechanisms: Implement safety mechanisms designed to handle the unique challenges posed by browser agents and their access to real-world systems. This could involve stronger input validation, restricting access to sensitive functions, enhanced monitoring, and better integration of safety models into the agent's decision-making process.
• Strengthen adversarial robustness: Develop more robust methods to resist jailbreaking attacks specifically targeting browser agents. This includes evaluating model behavior beyond simple prompt variations and addressing the implications of long contexts and action histories.
• Implement external sandboxing and monitoring: Operate browser agents within a secure sandbox environment with rigorous monitoring to detect and mitigate potentially harmful actions in real-time. Careful logging and analysis of agent behavior can be crucial for detecting and preventing future attacks