Language Model Security Database

The SMAB (Sensitivity-based Multi-Armed Bandit) framework introduces a vulnerability in text classifiers and Large Language Models (LLMs) by enabling efficient, black-box adversarial text generation. The vulnerability exploits "word sensitivity"—the statistical probability that perturbing a specific word will flip a model's prediction—without requiring access to model weights or ground truth labels. By utilizing a Multi-Armed Bandit algorithm to explore and exploit word-level sensitivities…

Large Language Models (LLMs) with refusal training are vulnerable to a "jailbreaking-to-jailbreak" (J2) attack. A J2 attack involves initially jailbreaking a powerful LLM to create a "J2 attacker." This attacker, instructed with general jailbreaking strategies, then autonomously attempts to jailbreak other LLMs, including potentially the same model it was derived from, by iteratively refining its attack based on previous attempts and in-context learning.

Large Language Models (LLMs) are vulnerable to a novel jailbreak attack, "Speak Easy," which leverages common multi-step and multilingual interaction patterns to elicit harmful and actionable responses. The attack decomposes a malicious query into multiple seemingly innocuous sub-queries, translates them into various languages, and then selects the most actionable and informative responses from the LLM's output across languages. This bypasses existing safety mechanisms more effectively than…

A vulnerability in Large Language Models (LLMs) allows adversarial reasoning attacks to bypass safety mechanisms and elicit harmful responses. The vulnerability stems from the insufficient robustness of existing LLM safety measures against iterative prompt refinement guided by a loss function that measures the LLM's proximity to generating a target harmful response. This allows an attacker to effectively navigate the prompt space, even against adversarially trained models, resulting in…

Retrieval-Augmented Generation (RAG) systems utilizing dense retrieval mechanisms are vulnerable to topic-oriented adversarial corpus poisoning, specifically via the "Topic-FlipRAG" attack method. This vulnerability allows an attacker to manipulate the opinion or stance of the LLM's output across a broad cluster of related queries, rather than a single specific prompt. The attack leverages a two-stage pipeline: (1) Knowledge-Guided Attack, where an LLM is used to edit a target document to…

A novel "Flanking Attack" exploits the vulnerability of multimodal LLMs (e.g., Google Gemini) to bypass content moderation filters by embedding adversarial prompts within a sequence of benign prompts. The attack leverages the LLM's processing of both audio and text, obfuscating harmful requests through contextualization and layering, thereby yielding policy-violating responses.

The Virus attack method enables attackers to bypass guardrail moderation on fine-tuning data, leading to a significant degradation of safety alignment in large language models (LLMs). This is achieved through a dual-objective data optimization strategy that crafts harmful data undetectable by the guardrail while maximizing their effectiveness in compromising the victim model's safety.

A vulnerability in several large language models (LLMs), including Qwen2.5-7BInstruct, Llama3.1-8B-Instruct, and GPT-4 variants, allows for black-box jailbreaking via prompt engineering techniques that exploit the proximity of benign and malicious prompt embeddings in the model's representation space. An attacker can craft prompts leveraging reinforcement learning to manipulate the embedding, causing the model to bypass its safety mechanisms and generate harmful or undesirable outputs while…

The GAP framework, as described in arXiv:2501.18638, reveals vulnerabilities in various large language models (LLMs) by generating stealthy jailbreak prompts that bypass content moderation systems. The framework leverages a graph-based attack strategy, enabling knowledge sharing across attack paths for enhanced efficiency and evasion. This allows the successful bypassing of multiple LLM safety mechanisms, including those based on perplexity and prompt-based heuristics.

Large Language Models (LLMs) used in hate speech detection systems are vulnerable to adversarial attacks and model stealing, resulting in evasion of hate speech detection. Adversarial attacks modify hate speech text to evade detection, while model stealing creates surrogate models that mimic the target system's behavior.