Language Model Security Database

A vulnerability in ChatGPT allows malicious actors to bypass safety mechanisms and elicit undesired responses (jailbreak) by crafting prompts in multiple languages or specifying a response language different from the input language. This is amplified by prompt injection techniques.

Large Language Models (LLMs) exhibit an inherent response tendency, predisposing them towards affirmation or rejection of instructions. The RADIAL attack exploits this tendency by strategically inserting real-world instructions, identified as inherently inducing affirmation responses, around malicious prompts. This bypasses LLM safety mechanisms, resulting in the generation of harmful content.

A vulnerability exists in large language models (LLMs) utilizing in-context learning (ICL). Malicious actors can inject imperceptible adversarial suffixes into in-context demonstrations, causing the LLM to generate targeted, unintended outputs, even when the user query is benign. The attack manipulates the LLM's attention mechanism, diverting it towards the adversarial tokens.

Large Language Model (LLM)-based agents, due to their multi-agent architecture and role-based interactions, are vulnerable to adversarial attacks that exploit the system's design and agent roles. Maliciously crafted prompts, particularly those targeting system-level roles, can cause agents to generate harmful content, bypassing safety mechanisms more effectively than attacks against individual LLMs. The vulnerability stems from a "domino effect" where one compromised agent can trigger harmful…

Large Language Models (LLMs) are vulnerable to jailbreaking attacks exploiting cognitive overload induced by multilingual prompts, veiled expressions, and effect-to-cause reasoning. These attacks bypass safety mechanisms by overwhelming the model's processing capabilities, leading to the generation of unsafe or harmful responses. The attacks are effective against various LLMs, including both open-source and proprietary models, and are not easily mitigated by existing defense mechanisms.

A prompt injection vulnerability in OpenAI's custom GPT models allows attackers to extract the system prompt and potentially leak user-uploaded files. Attackers craft malicious prompts that manipulate the LLM into revealing sensitive information, even when defensive prompts are in place. The vulnerability is exacerbated when the model includes a code interpreter.

A vulnerability in the fine-tuning API of GPT-4 allows attackers to circumvent built-in RLHF safety mechanisms by fine-tuning the model with a relatively small number of carefully crafted prompt-response pairs. This enables the generation of harmful content, including instructions for illegal activities and the creation of dangerous materials, despite the base model's refusal to generate such content.

A system prompt leakage vulnerability in GPT-4V allows extraction of internal system prompts through carefully crafted, incomplete conversations combined with image input. Extracted prompts can be used as highly effective jailbreak prompts, bypassing safety restrictions and leading to undesirable outputs, including revealing personally identifiable information from images.

Multimodal Large Language Models (MLLMs) are vulnerable to a novel attack vector where query-relevant images, generated using techniques like Stable Diffusion and typography, bypass safety mechanisms and elicit unsafe responses even when the underlying LLM is safety-aligned. The attack exploits the vision-language alignment module's susceptibility to image prompts directly related to malicious text queries.

Large Language Models (LLMs) are vulnerable to a targeted linguistic fuzzing attack that exploits the complexity of human language to bypass safety guardrails. The attack, termed "Jade," leverages transformational-generative grammar rules to systematically increase the syntactic complexity of benign seed questions, making them increasingly difficult for LLMs to recognize as malicious. This leads to the generation of unsafe content, even when the underlying semantics remain unchanged.