Language Model Security Database

Large Language Models (LLMs) are vulnerable to prompt extraction attacks via inversion of their normal outputs. An attacker can train a model to reconstruct the prompt used to generate multiple outputs from an LLM, even without access to internal model parameters (logits) or requiring adversarial queries. This allows extraction of both user and system prompts.

Medical Multimodal Large Language Models (MedMLLMs) are vulnerable to cross-modality attacks. Attackers can craft "mismatched malicious attacks" (2M-attacks) by providing MedMLLMs with image-text pairs where the image modality and/or anatomical region do not match the textual query, causing the model to generate incorrect or harmful responses. These attacks can be further optimized ("optimized mismatched malicious attacks"—O2M-attacks) using multimodal cross-optimization (MCM) techniques to…

A vulnerability in large language models (LLMs) allows attackers to elicit unsafe or unethical responses through a chain of semantically relevant multi-turn prompts. The attack, termed "Chain of Attack" (CoA), exploits the model's contextual understanding and adaptive response capabilities to gradually steer the conversation towards the desired harmful output, even if single-turn prompts are rejected due to safety mechanisms. The attack leverages semantic similarity scoring (e.g., using…

A vulnerability in large language models (LLMs) allows for near-perfect jailbreaking via iterative prompt refinement and self-explanation. The attacker uses the LLM itself to iteratively refine adversarial prompts by requesting self-explanations of failed attempts, ultimately generating prompts that bypass safety mechanisms and elicit harmful content. A subsequent "Rate+Enhance" step further maximizes the harmfulness of the generated output.

Large language models (LLMs) are vulnerable to enhanced jailbreak attacks by appending multiple end-of-sentence (EOS) tokens to malicious prompts. This bypasses internal safety mechanisms, causing the LLM to respond to harmful queries that it would otherwise reject. The EOS tokens subtly shift the LLM’s internal representation of the prompt, making it appear less harmful without significantly altering the semantic meaning of the malicious content.

A vulnerability in multimodal large language models (MLLMs) allows for efficient jailbreaking attacks by leveraging visual input to bypass safety mechanisms. The attack constructs a multimodal model by adding a visual module to the target LLM, then uses a modified PGD algorithm to optimize visual input to generate jailbreaking embeddings. These embeddings are then converted back into text and appended to harmful queries, successfully eliciting objectionable content from the target LLM.

Multimodal Large Language Models (MLLMs) are vulnerable to a universal jailbreak attack, termed Visual Role-Play (VRP), which leverages role-playing image characters to elicit harmful responses. VRP generates images depicting high-risk characters (e.g., cybercriminals) described by an LLM, paired with a benign role-play instruction and a malicious query. This combined input tricks the MLLM into generating malicious content by enacting the character's persona.

A vulnerability in the voice mode of GPT-4o allows bypassing safety restrictions through a novel "Voice Jailbreak" attack. This attack leverages principles of fictional storytelling (setting, character, plot) to craft audio prompts that persuade the LLM to generate responses violating OpenAI's usage policies, including generating content related to illegal activities, hate speech, physical harm, fraud, pornography, and privacy violations. The attack's success rate is significantly higher than…

Large Language Models (LLMs) are vulnerable to a novel jailbreaking attack, "WordGame," which leverages simultaneous query and response obfuscation to bypass safety mechanisms. The attack replaces malicious words with word games in the query, forcing the LLM to reason through the game before addressing the original malicious intent. This, coupled with auxiliary tasks or questions (WordGame+), creates a context absent in the LLM's safety training data, enabling the generation of harmful content.

Leading safety-aligned Large Language Models (LLMs) are vulnerable to simple adaptive jailbreaking attacks. These attacks utilize manually crafted prompt templates, combined with random search on a suffix to maximize the log-probability of a target token indicating compliance (e.g., "Sure"). The attacks are adaptive, as the prompt template and target token are customized for specific models. Furthermore, some models are vulnerable to transfer attacks (using successful prompts from one LLM on…