Language Model Security Database

Large Language Models (LLMs) exhibit increased vulnerability to adversarial prompts employing code-switching techniques, where multiple languages are interwoven within a single query. This vulnerability stems from an unintended correlation between the resource availability of the languages used in the prompt and the LLM's safety alignment. LLMs trained on imbalanced multilingual data are more susceptible to attacks leveraging low-resource languages, resulting in a higher rate of unsafe or…

Large Language Models (LLMs) are vulnerable to knowledge-based jailbreaks, where an attacker provides domain-specific knowledge to elicit harmful or unintended outputs. The vulnerability stems from the LLM's ability to process and respond to knowledge inputs in a way that circumvents safety mechanisms, even if the input knowledge itself isn't inherently malicious. Attackers leverage this by constructing prompts that combine seemingly innocuous knowledge with subtly manipulative phrasing to…

A deep reinforcement learning (DRL) based attack, termed RLbreaker, demonstrates the ability to more efficiently generate jailbreaking prompts for large language models (LLMs) than existing methods. The attack leverages a DRL agent to guide the search for effective prompt structures, bypassing safety mechanisms and eliciting undesirable responses to harmful questions. The effectiveness stems from the DRL agent's ability to strategically select prompt mutators, rather than relying on random…

Large Language Models (LLMs) are vulnerable to jailbreak attacks exploiting uncommon text-encoded structures (UTES) rarely encountered during training. These UTES, such as JSON, tree representations, or LaTeX code, embedded within prompts, cause LLMs to bypass safety mechanisms and generate harmful content. The attack's success stems from the LLM's difficulty in processing and interpreting these unusual structures, coupled with the obfuscation of malicious instructions within the structured…

Large Language Models (LLMs) used to control robots exhibit biases leading to discriminatory and unsafe behaviors. When provided with personal characteristics (e.g., race, gender, disability), LLMs generate biased outputs resulting in discriminatory actions (e.g., assigning lower rescue priority to certain groups) and accept or deem feasible dangerous or unlawful instructions (e.g., removing a person's mobility aid).

RL-JACK is a reinforcement learning-based black-box attack that generates jailbreaking prompts to bypass safety mechanisms in LLMs. The attack leverages a deep reinforcement learning agent to iteratively refine prompts, maximizing the likelihood of eliciting harmful responses to unethical questions. The effectiveness stems from a novel reward function that provides continuous feedback based on cosine similarity to a reference answer from an unaligned LLM, and an action space that strategically…

GPT-4o exhibits vulnerability to jailbreak attacks via audio prompts, despite enhanced safety against text-based attacks. Successful jailbreaks can be achieved by converting text prompts, including those optimized for adversarial attacks against other LLMs (demonstrated using GCG, AutoDAN, PAP, and BAP methods), into audio using text-to-speech (TTS) synthesis. This circumvention allows elicitation of unsafe responses from GPT-4o that would otherwise be prevented by its safety mechanisms. The…

Large Vision Language Models (LVLMs) are vulnerable to a bi-modal adversarial prompt attack (BAP). BAP leverages a combined textual and visual prompt to bypass safety mechanisms and elicit harmful responses, even in models designed to resist single-modality attacks. The attack first introduces a query-agnostic adversarial perturbation to the visual prompt, making the model more likely to respond positively regardless of the text. Then, an LLM refines the textual prompt iteratively to achieve…

Large Language Models (LLMs) are vulnerable to a black-box query-response optimization attack (QROA). QROA iteratively refines a malicious prompt suffix using a surrogate model to maximize a reward function that measures the likelihood of eliciting harmful content from the LLM. This attack does not require access to the model's internal parameters or logits; it operates solely via standard query-response interactions.

A vulnerability in aligned Large Language Models (LLMs) allows circumvention of safety mechanisms through improved few-shot jailbreaking techniques. The attack leverages injection of special system tokens (e.g., [/INST]) into few-shot demonstrations and demo-level random search to optimize the probability of generating harmful responses. This bypasses defenses that rely on perplexity filtering and input perturbation.