Language Model Security Database

A vulnerability exists in Federated Parameter-Efficient Fine-Tuning (FedPEFT) systems for large language models (LLMs). Malicious clients can exploit the PEFT mechanism (e.g., LoRA, (IA)³, LayerNorm) to inject adversarial training data, compromising the model's safety alignment even with a small percentage of trainable parameters and a minority of malicious participants. The attack, termed "PEFT-as-an-Attack" (PaaA), circumvents the LLM's safety guardrails, causing it to generate harmful…

A vulnerability exists in several Large Vision-Language Models (LVLMs) where seemingly safe images, when combined with additional safe images and prompts using a specific attack methodology (Safety Snowball Agent), can trigger the generation of unsafe and harmful content. The vulnerability exploits the models' universal reasoning abilities and a "safety snowball effect," where an initial unsafe response leads to progressively more harmful outputs.

Large Language Models (LLMs) are vulnerable to a jailbreak attack that leverages the model's ability to generate diverse and obfuscated prompts to bypass safety constraints. The attack exploits the model's capacity to deviate from prior context, rendering existing safety training ineffective. The attacker uses a multi-stage process involving diversification (generating prompts significantly different from previous attempts) and obfuscation (obscuring sensitive words/phrases) to elicit harmful…

Large Language Models (LLMs) are vulnerable to jailbreak attacks using language games, which manipulate input prompts through structured linguistic alterations (e.g., Ubbi Dubbi, custom letter insertion rules) to bypass safety mechanisms. These games obfuscate malicious intent while maintaining human readability, causing LLMs to generate unsafe content.

Large Language Models (LLMs) are vulnerable to multi-round jailbreak attacks which leverage a heuristic search process to progressively elicit harmful content. The attack decomposes a harmful query into multiple, seemingly innocuous sub-queries, iteratively refining the prompts based on the LLM's responses and employing psychological strategies to bypass safety mechanisms. This allows for the circumvention of single-round detection methods and elicitation of responses containing prohibited…

Large Language Models (LLMs) are vulnerable to multi-step "moralized" jailbreak prompts that bypass their safety guardrails. These prompts, while appearing ethical individually, cumulatively create a context that elicits verbally aggressive and harmful content generation. The attack leverages the LLMs' inability to fully understand the cumulative context and intent across multiple prompts.

Large language models (LLMs) are vulnerable to jailbreak attacks exploiting nonlinear features within prompt encodings. These features, not detectable by linear methods, allow adversaries to reliably elicit harmful outputs despite safety training. Different attack methods leverage distinct nonlinear features, limiting the transferability of detection and mitigation techniques.

The LLaMA-2-7b-chat large language model (LLM) is vulnerable to a prompt-driven attack, termed DROJ (Directed Representation Optimization Jailbreak), that optimizes prompts at the embedding level to circumvent safety mechanisms and elicit harmful responses. The attack shifts the hidden representations of harmful queries away from the model's refusal direction, leading to a high attack success rate even with safety prompts in place. While the model may not refuse, responses may be repetitive…

LLMStinger uses a reinforcement-learning loop to fine-tune an attacker model that generates adversarial suffixes for jailbreak prompts. Because the approach does not require white-box access to the target, it can adapt existing attacks for both open- and closed-source assistants and bypass otherwise effective refusal behavior.

A novel SQL Injection Jailbreak (SIJ) vulnerability allows attackers to bypass safety mechanisms in Large Language Models (LLMs) by manipulating the structure of input prompts. The attack leverages the model's processing of system prompts, user prefixes, user prompts, and assistant prefixes to effectively "comment out" the expected response prefix and inject harmful instructions, causing the LLM to generate unsafe content. This vulnerability exploits the external properties of the LLM…