Language Model Security Database

A vulnerability in several large language models (LLMs), including Qwen2.5-7BInstruct, Llama3.1-8B-Instruct, and GPT-4 variants, allows for black-box jailbreaking via prompt engineering techniques that exploit the proximity of benign and malicious prompt embeddings in the model's representation space. An attacker can craft prompts leveraging reinforcement learning to manipulate the embedding, causing the model to bypass its safety mechanisms and generate harmful or undesirable outputs while…

The GAP framework, as described in arXiv:2501.18638, reveals vulnerabilities in various large language models (LLMs) by generating stealthy jailbreak prompts that bypass content moderation systems. The framework leverages a graph-based attack strategy, enabling knowledge sharing across attack paths for enhanced efficiency and evasion. This allows the successful bypassing of multiple LLM safety mechanisms, including those based on perplexity and prompt-based heuristics.

Large Language Models (LLMs) used in hate speech detection systems are vulnerable to adversarial attacks and model stealing, resulting in evasion of hate speech detection. Adversarial attacks modify hate speech text to evade detection, while model stealing creates surrogate models that mimic the target system's behavior.

Large Language Models (LLMs) including GPT-4o, LLaMA 3.2, and others exhibit a vulnerability to "Task-in-Prompt" (TIP) adversarial attacks. This vulnerability allows attackers to bypass safety alignment and content filtering mechanisms by embedding prohibited instructions within benign sequence-to-sequence tasks (such as ciphers, riddles, code execution, or text transformation). The model implicitly decodes the obfuscated content via self-attention mechanisms during token generation…

Large Language Models (LLMs) are vulnerable to multi-turn adversarial attacks that skillfully decompose malicious requests into seemingly benign interactions, progressively guiding the dialogue towards harmful outputs. This vulnerability allows attackers to bypass LLM safety mechanisms through a series of strategically crafted prompts, exploiting the model's iterative response generation. The attack's success hinges on dynamically adapting each prompt based on the LLM's previous responses…

End-to-end Large Audio Language Models (LALMs) contain an audio-based jailbreak vulnerability allowing attackers to bypass safety alignment guardrails by manipulating audio-specific "hidden semantics." Unlike text-based attacks, this exploitation involves encoding harmful queries into audio and applying signal processing modifications—specifically changes to emphasis, speech speed, intonation, tone, background noise, celebrity accents, or emotional overlays (e.g., laughter, screaming). These…

Large Language Models (LLMs) are vulnerable to malicious prompts disguised as summaries of scientific papers, even when those papers are fabricated by the attacker. This allows attackers to manipulate LLMs into generating responses exhibiting significantly increased stereotypical bias and toxicity. The vulnerability is exacerbated by multi-turn interactions, where bias scores tend to increase with each subsequent response. The inclusion of author names and publication venues in the fabricated…

Large language models (LLMs) exhibit increased responsiveness to prompts framed within positive narratives. The Happy Ending Attack (HEA) exploits this by embedding malicious requests within a positive-sentiment scenario culminating in a happy ending. This allows the LLM to generate responses that fulfill the malicious request while perceiving the overall prompt as benign.

Large Language Models (LLMs) are vulnerable to a self-instruct few-shot jailbreaking attack that leverages pattern and behavior learning to bypass safety mechanisms. The attack efficiently induces harmful outputs by injecting a strategically chosen response prefix into the model's prompt and exploiting the model's tendency to mimic co-occurrence patterns of special tokens preceding the prefix. This allows the attacker to elicit unsafe responses with a small number of carefully crafted…

Large language models (LLMs) are vulnerable to adversarial prompt engineering attacks that leverage proximity constraints to elicit harmful behaviors. By subtly modifying benign prompts within a semantically close embedding space, attackers can bypass existing safety mechanisms and induce undesired outputs, even when the original prompts would not trigger such a response. This vulnerability exploits the model's sensitivity to small perturbations in the input embedding, resulting in the…