Language Model Security Database

Large Language Models (LLMs) used as safety judges are vulnerable to an "Emoji Attack," a prompt injection technique that leverages token segmentation bias. Inserting emojis within tokens alters sub-token embeddings, misleading the judge LLM into classifying harmful content as safe. The attack's effectiveness is amplified by strategically placing emojis to maximize the embedding discrepancy between sub-tokens and the original token.

Large Language Models (LLMs) are vulnerable to transferable ensemble black-box jailbreak attacks. The vulnerability allows an attacker to bypass safety mechanisms and elicit undesired or harmful responses from the LLM by using an ensemble of LLM-as-attacker methods that optimize malicious prompts, adaptively adjusting resources based on prompt difficulty, and strategically modifying prompt semantics to evade detection.

Large Language Models (LLMs) are vulnerable to jailbreaking via the addition of adversarial suffixes generated by models like AmpleGCG-Plus. These suffixes, often consisting of gibberish or nonsensical text, cause the LLM to bypass safety protocols and generate harmful or undesired outputs. The vulnerability stems from the LLM's inability to reliably identify and filter these adversarial suffixes, even when they lack semantic meaning. AmpleGCG-Plus significantly improves the success rate and…

Large Vision-Language Models (VLMs) are vulnerable to a novel black-box jailbreak attack, IDEATOR, which leverages a separate VLM to generate malicious image-text pairs. The attacker VLM iteratively refines its prompts based on the target VLM's responses, bypassing safety mechanisms by generating contextually relevant and visually subtle malicious prompts.

Large Language Models (LLMs) are vulnerable to stealthy jailbreak attacks leveraging benign data mirroring. Attackers train a local "mirror model" on benign data obtained from the target LLM. This mirror model, mimicking the target's behavior, is then used to generate adversarial prompts, which are subsequently deployed against the target LLM, bypassing content moderation systems due to the lack of overtly malicious content in the initial data gathering phase.

Large Language Models (LLMs) are vulnerable to a novel iterative self-tuning attack (ADV-LLM) that crafts adversarial suffixes. This attack significantly reduces the computational cost of generating effective jailbreaks compared to prior methods, achieving near 100% success rate against various open-source LLMs and high success rates (e.g., 99% against GPT-3.5, 49% against GPT-4) against closed-source models. The attack leverages iterative self-tuning to improve the LLM's ability to generate…

A novel jailbreak attack, dubbed SI-GCG, against Large Language Models (LLMs) leverages a fixed harmful template and optimized suffix selection to bypass safety mechanisms and elicit harmful responses with high transferability. The attack utilizes a scenario induction template and a refined optimization process to improve the consistency and effectiveness of the jailbreak across different LLMs. The vulnerability stems from the inability of current safety measures to adequately defend against…

Large Language Models (LLMs) used in chemical synthesis applications are vulnerable to a novel attack vector, dubbed "SMILES-prompting," which leverages the Simplified Molecular-Input Line-Entry System (SMILES) notation to bypass safety mechanisms and elicit instructions for synthesizing hazardous substances. The attack exploits the LLM's inability to effectively filter or interpret SMILES strings representing dangerous chemicals, leading to the disclosure of synthesis procedures.

Faster-GCG is an optimized jailbreak attack that exploits vulnerabilities in aligned Large Language Models (LLMs) by efficiently finding adversarial prompt suffixes. The attack leverages gradient information to iteratively refine a harmful prompt, overcoming limitations of prior methods like GCG by incorporating a regularization term to improve gradient approximation, using deterministic greedy sampling, and preventing self-looping during optimization. This allows for significantly higher…

Large Language Model (LLM) agents are vulnerable to obfuscated adversarial prompts that exploit tool misuse. These prompts, crafted through prompt optimization techniques, force the agent to execute tools (e.g., URL fetching, markdown rendering) in a way that leaks sensitive user data (e.g., PII) without the user's knowledge. The prompts are designed to be visually indistinguishable from benign prompts.