Language Model Security Database

Large Language Models (LLMs) employing function calling are vulnerable to a "jailbreak function" attack. Maliciously crafted function definitions and prompts can coerce the LLM into generating harmful content within the function's arguments, bypassing existing safety filters designed for chat modes. This exploits discrepancies in safety alignment between function argument generation and chat response generation.

Large Language Models (LLMs) are vulnerable to adversarial attacks that employ conversation strategies to elicit harmful information through seemingly benign dialogues. The attack, termed "Imposter.AI," leverages three key strategies: (1) decomposing malicious questions into innocuous sub-questions; (2) rephrasing overtly malicious questions into benign-sounding alternatives; and (3) enhancing the harmfulness of responses by prompting the LLM for illustrative examples. This allows attackers to…

Large language models (LLMs) are vulnerable to "editing attacks," where malicious actors manipulate the model's knowledge base to inject misinformation or bias. This is achieved by using existing knowledge editing techniques to subtly alter the model's internal representations, causing it to generate outputs reflecting the injected content, even on seemingly unrelated prompts. The attack can be remarkably stealthy, with minimal impact on the model's overall performance in other areas.

Large Language Models (LLMs) struggle to generate genuinely fallacious reasoning. When prompted to create a false procedure for a harmful task, the LLMs instead leak the correct, harmful procedure while incorrectly claiming it's false. This vulnerability allows bypassing safety mechanisms and eliciting harmful outputs.

Large Language Models (LLMs) are vulnerable to adversarial attacks that utilize low-perplexity prompts to elicit unsafe content. These prompts, while statistically likely to occur in normal conversation, can trigger the generation of harmful or toxic outputs that evade standard safety filters. The vulnerability stems from the model's inability to reliably distinguish between benign and malicious intents within the statistical distribution of natural language.

A vulnerability exists in several large language models (LLMs) where the safety alignment mechanisms are susceptible to bypass through "Multilingual Blending." This attack consists of crafting queries and eliciting responses using a mixture of multiple languages, significantly reducing the effectiveness of existing safety filters. The vulnerability stems from the models' ability to process and generate text in multiple languages, which, when combined in specific ways, can confuse the safety…

The Automated Progressive Red Teaming (APRT) framework exploits vulnerabilities in large language models (LLMs) by iteratively generating adversarial prompts. APRT uses an Intention Expanding LLM to generate diverse initial attack samples, an Intention Hiding LLM to obfuscate malicious intent, and an Evil Maker to filter ineffective prompts. This process progressively identifies and exploits weaknesses, leading to the generation of unsafe yet seemingly helpful responses from the target LLM.

The SoP framework allows for automated generation of jailbreak prompts, bypassing safety mechanisms in LLMs. SoP utilizes multiple automatically optimized "jailbreak characters" within a single prompt to persuade the LLM to generate harmful or undesirable content, even without any seed jailbreak templates. This vulnerability is demonstrated against GPT-3.5, GPT-4, and LLaMA-2.

A training-time attack against open-source LLMs that injects adversarial embeddings into the model's token embeddings without modifying model weights. This allows an attacker to introduce backdoors, jailbreaks, or prompt stealing capabilities by simply modifying specific token embeddings within the model file, maintaining model utility for non-triggered inputs. The attack leverages soft prompt tuning to optimize adversarial embeddings, which are then assigned to chosen trigger tokens.

Appending a single whitespace character (space) or certain punctuation marks to the end of an LLM's input template can bypass safety mechanisms and cause the model to generate unsafe, biased, or factually incorrect outputs, even if the original prompt was benign. This vulnerability is due to the statistical properties of single-character tokens in the model's training data, causing unintended behavior in the model's token prediction.