Language Model Security Database

Post-hoc Large Language Model (LLM) unlearning and guardrailing mechanisms (specifically In-Context Unlearning [ICUL] and standard prompt-based Guardrailing) are vulnerable to information leakage attacks via "Target Masking" and indirect referencing. These systems rely on superficial semantic matching to suppress "forget sets" (specific entities or concepts). Attackers can bypass these restrictions by querying associated properties, relationships, or pseudonyms rather than the explicit target…

State-of-the-art machine unlearning and safety fine-tuning methods for Large Language Models (LLMs) fail to robustly remove hazardous capabilities or refusal mechanisms from model weights. While these methods suppress model outputs during standard input-output interactions, the underlying capabilities remain latent in the parameter space. An attacker with access to model weights (e.g., via open releases or leaked weights) can restore "unlearned" knowledge (such as dual-use biology hazards) or…

The SMAB (Sensitivity-based Multi-Armed Bandit) framework introduces a vulnerability in text classifiers and Large Language Models (LLMs) by enabling efficient, black-box adversarial text generation. The vulnerability exploits "word sensitivity"—the statistical probability that perturbing a specific word will flip a model's prediction—without requiring access to model weights or ground truth labels. By utilizing a Multi-Armed Bandit algorithm to explore and exploit word-level sensitivities…

The Emoti-Attack vulnerability constitutes a zero-word-perturbation adversarial attack against Natural Language Processing (NLP) systems and Large Language Models (LLMs). The vulnerability exploits the discrete embedding space of emojis and emoticons to manipulate model behavior without altering the semantic content or character integrity of the original text. By appending strategically optimized emoji sequences to the prefix and suffix of an input string (formalized as $s \oplus x \oplus…

Vision Language Models (VLMs) integrated into autonomous driving (AD) systems are vulnerable to a black-box adversarial attack method termed Cascading Adversarial Disruption (CAD). The vulnerability stems from the model's susceptibility to optimized visual perturbations that disrupt the decision-making reasoning chain (perception, prediction, and planning). Attackers can generate adversarial images or physical patches by aligning visual noise with deceptive textual semantics in the model's…

A multi-step prompt injection vulnerability allows attackers to bypass Large Language Model (LLM) safety guardrails by combining prompt obfuscation with task decomposition. The attack methodology, identified as part of the CySecBench research, employs a "Word Reversal" technique where every fifth word in the malicious input is reversed to evade initial keyword detection. This obfuscated input is then embedded within a benign educational context, specifically instructing the model to act as a…

A vulnerability in several large language models (LLMs), including Qwen2.5-7BInstruct, Llama3.1-8B-Instruct, and GPT-4 variants, allows for black-box jailbreaking via prompt engineering techniques that exploit the proximity of benign and malicious prompt embeddings in the model's representation space. An attacker can craft prompts leveraging reinforcement learning to manipulate the embedding, causing the model to bypass its safety mechanisms and generate harmful or undesirable outputs while…

The GAP framework, as described in arXiv:2501.18638, reveals vulnerabilities in various large language models (LLMs) by generating stealthy jailbreak prompts that bypass content moderation systems. The framework leverages a graph-based attack strategy, enabling knowledge sharing across attack paths for enhanced efficiency and evasion. This allows the successful bypassing of multiple LLM safety mechanisms, including those based on perplexity and prompt-based heuristics.

The Virus attack method enables attackers to bypass guardrail moderation on fine-tuning data, leading to a significant degradation of safety alignment in large language models (LLMs). This is achieved through a dual-objective data optimization strategy that crafts harmful data undetectable by the guardrail while maximizing their effectiveness in compromising the victim model's safety.

Large language models (LLMs) exhibit increased responsiveness to prompts framed within positive narratives. The Happy Ending Attack (HEA) exploits this by embedding malicious requests within a positive-sentiment scenario culminating in a happy ending. This allows the LLM to generate responses that fulfill the malicious request while perceiving the overall prompt as benign.