Language Model Security Database

Large Language Models (LLMs) are vulnerable to an adversarial encoding attack where English instructions are obfuscated using valid but visually nonsensical UTF-8 byte sequences. By manipulating multi-byte UTF-8 encoding schemes—specifically by fixing the last 8 bits of a code point to match a target ASCII character and rotating the remaining bits—attackers can generate sequences (e.g., Byzantine musical symbols) that appear incomprehensible to humans and standard text filters but are…

A vulnerability exists in Large Language Models (LLMs) that allows for efficient jailbreaking by selectively fine-tuning only the lower layers of the model with a toxic dataset. This "Freeze Training" method, as described in the research paper, concentrates the fine-tuning on layers identified as being highly sensitive to the generation of harmful content. This approach significantly reduces training duration and GPU memory consumption while maintaining a high jailbreak success rate.

LLM agents utilizing external tools are vulnerable to indirect prompt injection (IPI) attacks. Attackers can embed malicious instructions into the external data accessed by the agent, manipulating its behavior even when defenses against direct prompt injection are in place. Adaptive attacks, which modify the injected payload based on the specific defense mechanism, consistently bypass existing defenses with a success rate exceeding 50%.

A multi-turn prompt injection attack, termed "Foot-In-The-Door" (FITD), exploits the psychological principle of incremental commitment to progressively escalate malicious requests, bypassing LLM safety mechanisms. The attack leverages intermediate "bridge" prompts and self-alignment techniques to coax the model into generating increasingly harmful outputs, even when initially refusing similar direct requests.

Large Language Models (LLMs) are vulnerable to a novel jailbreaking attack leveraging adversarial metaphors. The attack, termed AVATAR, induces the LLM to reason about benign metaphors related to harmful tasks, ultimately leading to the generation of harmful content either directly or through calibration of metaphorical and professional harmful content. The attack exploits the LLM's cognitive mapping process, bypassing standard safety mechanisms.

The Emoti-Attack vulnerability constitutes a zero-word-perturbation adversarial attack against Natural Language Processing (NLP) systems and Large Language Models (LLMs). The vulnerability exploits the discrete embedding space of emojis and emoticons to manipulate model behavior without altering the semantic content or character integrity of the original text. By appending strategically optimized emoji sequences to the prefix and suffix of an input string (formalized as $s \oplus x \oplus…

Large Language Models (LLMs) are vulnerable to jailbreaking attacks leveraging mutation-based fuzzing techniques. The TurboFuzzLLM framework efficiently generates adversarial prompts, combining mutated templates with harmful questions to elicit unauthorized or malicious responses. This vulnerability allows bypassing built-in safeguards and obtaining harmful outputs through black-box API access. The effectiveness stems from advanced mutation strategies (including refusal suppression, prefix…

Large Reasoning Models (LRMs) are vulnerable to a novel jailbreak attack, "Mousetrap," which leverages the models' reasoning capabilities to elicit harmful responses. Mousetrap uses a "Chaos Machine" to iteratively transform prompts via one-to-one mappings (e.g., character substitutions, word reversals), creating complex reasoning chains that confuse the LRM and cause it to generate unsafe outputs despite safety mechanisms. The iterative nature of the attack, combined with role-playing…

Large Language Models (LLMs) with structured output interfaces are vulnerable to jailbreak attacks that exploit the interaction between token-level inference and sentence-level safety alignment. Attackers can manipulate the model's output by constructing attack patterns based on prefixes of safety refusal responses and desired harmful outputs, effectively bypassing safety mechanisms through iterative API calls and constrained decoding. This allows the generation of harmful content despite…

Vision-Language Models (VLMs), specifically the LLaVA-1.5 and LLaVA-1.6 series, are vulnerable to optimization-based white-box jailbreak attacks despite standard safety alignment measures like Supervised Fine-Tuning (SFT) and Direct Preference Optimization (DPO). Attackers can craft adversarial perturbations in the image space (imperceptible noise) or latent space using Projected Gradient Descent (PGD) to manipulate the model's internal representations. These perturbations maximize the…