Language Model Security Database

Large Language Models (LLMs) are vulnerable to In-Context Attacks (ICA) and susceptible to mitigation via In-Context Defense (ICD). ICA leverages a small number of harmful demonstration examples within a prompt to elicit harmful responses from the LLM, even if it is otherwise safety-aligned. ICD counteracts ICA by prepending safe demonstration examples to the prompt, effectively reducing the likelihood of harmful output. The effectiveness of both ICA and ICD is demonstrated across multiple LLMs.

Large Language Models (LLMs) employing alignment techniques remain vulnerable to "jailbreak" attacks. The AutoDAN technique automatically generates semantically meaningful prompts that bypass safety features and elicit malicious outputs from aligned LLMs, unlike previous methods producing nonsensical prompts easily detectable by perplexity checks. These prompts exploit weaknesses in the LLM's alignment, causing it to generate responses that violate intended safety constraints.

Large Language Models (LLMs), such as GPT-4, exhibit a cross-lingual vulnerability in their safety mechanisms. Translating unsafe English prompts into low-resource languages, using readily available translation APIs like Google Translate, bypasses the LLM's safety filters and elicits harmful responses with a significantly higher success rate than attacks targeting the English language directly. The vulnerability stems from an unequal distribution of safety training data across languages…

Large Language Models (LLMs) are susceptible to automated jailbreak attacks using a fuzzing framework that generates variations of existing jailbreak prompts. This vulnerability allows bypassing built-in safety mechanisms, leading to the generation of harmful or unintended outputs. The vulnerability stems from the LLMs' inability to consistently recognize and reject semantically similar, but subtly different prompt variations generated through automated mutation techniques.

A universal black-box jailbreaking vulnerability exists in Large Language Models (LLMs) due to their susceptibility to adversarial prompts crafted using a genetic algorithm (GA). The GA optimizes a universal adversarial prompt suffix that, when appended to various user inputs, causes the LLM to generate unintended and potentially harmful outputs, bypassing safety mechanisms. This attack requires no knowledge of the LLM's internal architecture or parameters.

Large Language Models (LLMs) are vulnerable to a "Chain of Utterances" (CoU) based prompt injection attack. This attack exploits the LLM's ability to engage in multi-turn conversations and role-playing, tricking it into providing harmful or unsafe responses even when presented with safety guidelines. The attack leverages a crafted conversation between two agents ("Red-LM," a malicious agent, and "Base-LM," a seemingly helpful agent) to elicit unethical responses from the Base-LM by subtly…

Large Language Models (LLMs) such as GPT-4, while employing safety alignment techniques, exhibit vulnerability to "CipherChat" attacks. CipherChat leverages cipher prompts (e.g., ASCII, Unicode, Caesar cipher, Morse code) combined with system role descriptions and few-shot enciphered demonstrations to bypass safety mechanisms trained on natural language. This allows an attacker to elicit unsafe responses from the LLM, effectively evading safety filters. The vulnerability is amplified by the…

Aligned large language models (LLMs) are vulnerable to a universal and transferable adversarial suffix attack. Appending a specific, automatically generated suffix to a wide range of prompts, even those requesting objectionable content, causes the models to generate harmful or objectionable responses instead of refusing the request. The attack's success rate is significantly higher on GPT-based models.

The MASTER KEY framework exploits timing-based characteristics of Large Language Model (LLM) chatbot responses to infer internal defense mechanisms and automatically generate jailbreak prompts. This allows bypassing safety restrictions and eliciting responses violating usage policies, including generation of illegal, harmful, privacy-violating, and adult content. The framework utilizes a three-step process: reverse-engineering defenses via time-based analysis, creating proof-of-concept…

Large Language Models (LLMs), specifically ChatGPT versions 3.5 and 4.0, are vulnerable to prompt engineering attacks that circumvent built-in content restrictions. Attackers can craft malicious prompts, categorized into "pretending," "attention shifting," and "privilege escalation" techniques, to elicit responses containing prohibited content (e.g., instructions for illegal activities, generation of harmful content). The vulnerability stems from the LLM's inability to reliably distinguish…