Language Model Security Database

A deep reinforcement learning (DRL) based attack, termed RLbreaker, demonstrates the ability to more efficiently generate jailbreaking prompts for large language models (LLMs) than existing methods. The attack leverages a DRL agent to guide the search for effective prompt structures, bypassing safety mechanisms and eliciting undesirable responses to harmful questions. The effectiveness stems from the DRL agent's ability to strategically select prompt mutators, rather than relying on random…

Large Language Models (LLMs) are vulnerable to jailbreak attacks exploiting uncommon text-encoded structures (UTES) rarely encountered during training. These UTES, such as JSON, tree representations, or LaTeX code, embedded within prompts, cause LLMs to bypass safety mechanisms and generate harmful content. The attack's success stems from the LLM's difficulty in processing and interpreting these unusual structures, coupled with the obfuscation of malicious instructions within the structured…

Large Language Models (LLMs) used to control robots exhibit biases leading to discriminatory and unsafe behaviors. When provided with personal characteristics (e.g., race, gender, disability), LLMs generate biased outputs resulting in discriminatory actions (e.g., assigning lower rescue priority to certain groups) and accept or deem feasible dangerous or unlawful instructions (e.g., removing a person's mobility aid).

RL-JACK is a reinforcement learning-based black-box attack that generates jailbreaking prompts to bypass safety mechanisms in LLMs. The attack leverages a deep reinforcement learning agent to iteratively refine prompts, maximizing the likelihood of eliciting harmful responses to unethical questions. The effectiveness stems from a novel reward function that provides continuous feedback based on cosine similarity to a reference answer from an unaligned LLM, and an action space that strategically…

Large Vision Language Models (LVLMs) are vulnerable to a bi-modal adversarial prompt attack (BAP). BAP leverages a combined textual and visual prompt to bypass safety mechanisms and elicit harmful responses, even in models designed to resist single-modality attacks. The attack first introduces a query-agnostic adversarial perturbation to the visual prompt, making the model more likely to respond positively regardless of the text. Then, an LLM refines the textual prompt iteratively to achieve…

Large Language Models (LLMs) are vulnerable to a black-box query-response optimization attack (QROA). QROA iteratively refines a malicious prompt suffix using a surrogate model to maximize a reward function that measures the likelihood of eliciting harmful content from the LLM. This attack does not require access to the model's internal parameters or logits; it operates solely via standard query-response interactions.

A vulnerability in aligned Large Language Models (LLMs) allows circumvention of safety mechanisms through improved few-shot jailbreaking techniques. The attack leverages injection of special system tokens (e.g., [/INST]) into few-shot demonstrations and demo-level random search to optimize the probability of generating harmful responses. This bypasses defenses that rely on perplexity filtering and input perturbation.

Large Language Models (LLMs) are vulnerable to improved optimization-based jailbreaking attacks. The vulnerability stems from the susceptibility of LLMs to crafted prompts that exploit weaknesses in their safety mechanisms, allowing them to generate harmful responses despite safety training. This vulnerability is exacerbated by the use of diverse target templates containing harmful self-suggestions and guidance within the optimization goal, accelerating the convergence of the attack.

Large language models (LLMs) are vulnerable to enhanced jailbreak attacks by appending multiple end-of-sentence (EOS) tokens to malicious prompts. This bypasses internal safety mechanisms, causing the LLM to respond to harmful queries that it would otherwise reject. The EOS tokens subtly shift the LLM’s internal representation of the prompt, making it appear less harmful without significantly altering the semantic meaning of the malicious content.

A vulnerability allows attackers to bypass Large Language Model (LLM) moderation guardrails by using specially crafted prompts containing "cipher characters." These characters, strategically placed within the prompt's output, alter the LLM's response to reduce its "harm" score, enabling the generation of content that would otherwise be blocked. The attack leverages a jailbreak prefix combined with a malicious question and cipher characters to bypass both input and output level filters. This…