Language Model Security Database

Refusal-trained Large Language Models (LLMs) show decreased safety when deployed as browser agents compared to their performance in chatbot settings. Attack methods effective at jailbreaking LLMs in chat contexts also successfully bypass safety mechanisms in browser agents, leading to the execution of harmful behaviors. This vulnerability stems from a lack of generalization of safety training to agentic, real-world interaction scenarios and the increased context available to the agent (browser…

Large Language Models (LLMs) used in role-playing systems are vulnerable to character hallucination attacks, a form of jailbreak exploiting "query sparsity" and "role-query conflict". Query sparsity occurs when prompts fall outside the model's training data distribution, causing it to generate out-of-character responses. Role-query conflict arises when the prompt contradicts the established character persona, leading to inconsistent behavior. These vulnerabilities allow attackers to elicit…

Jailbreaking vulnerabilities in Large Language Models (LLMs) used in Retrieval-Augmented Generation (RAG) systems allow escalation of attacks from entity extraction to full document extraction and enable the propagation of self-replicating malicious prompts ("worms") within interconnected RAG applications. Exploitation leverages prompt injection to force the LLM to return retrieved documents or execute malicious actions specified within the prompt.

A vulnerability allows bypassing safety filters in text-to-image (T2I) models using a multi-agent framework ("Atlas") powered by Large Language Models (LLMs). Atlas iteratively generates and refines prompts, leveraging a Vision-Language Model (VLM) to assess filter activation and an LLM to select effective prompts that maintain semantic similarity to the original, malicious prompt while evading the filter. This enables the generation of images containing unsafe content.

LLM-based autonomous agents are vulnerable to malfunction amplification attacks. These attacks exploit the inherent instability of agents by inducing repetitive or irrelevant actions through various methods including prompt injection and adversarial perturbations, leading to agent malfunction and task failure. The attacks do not rely on overtly harmful actions, making them harder to detect with standard LLM safety mechanisms.

Embodied Large Language Models (LLMs) are vulnerable to manipulation via voice-based interactions, leading to the execution of harmful physical actions. Attacks exploit three vulnerabilities: (1) cascading LLM jailbreaks resulting in malicious robotic commands; (2) misalignment between linguistic outputs (verbal refusal) and physical actions (command execution); and (3) conceptual deception, where seemingly benign instructions lead to harmful outcomes due to incomplete world knowledge within…

The SoP framework allows for automated generation of jailbreak prompts, bypassing safety mechanisms in LLMs. SoP utilizes multiple automatically optimized "jailbreak characters" within a single prompt to persuade the LLM to generate harmful or undesirable content, even without any seed jailbreak templates. This vulnerability is demonstrated against GPT-3.5, GPT-4, and LLaMA-2.

LLMs, even when individually assessed as "safe," can be combined by an adversary to achieve malicious outcomes. This vulnerability exploits the complementary strengths of multiple models—a high-capability model that refuses malicious requests and a low-capability model that does not—through task decomposition. Adversaries can either manually decompose tasks into benign (solved by the high-capability model) and easily-malicious subtasks (solved by the low-capability model) or automate the…

A vulnerability in multimodal large language models (MLLMs) allows for efficient jailbreaking attacks by leveraging visual input to bypass safety mechanisms. The attack constructs a multimodal model by adding a visual module to the target LLM, then uses a modified PGD algorithm to optimize visual input to generate jailbreaking embeddings. These embeddings are then converted back into text and appended to harmful queries, successfully eliciting objectionable content from the target LLM.

A vulnerability in large language models (LLMs) allows attackers to elicit unsafe or unethical responses through a chain of semantically relevant multi-turn prompts. The attack, termed "Chain of Attack" (CoA), exploits the model's contextual understanding and adaptive response capabilities to gradually steer the conversation towards the desired harmful output, even if single-turn prompts are rejected due to safety mechanisms. The attack leverages semantic similarity scoring (e.g., using…