Prompt layer Vulnerabilities

Large Language Models (LLMs) are vulnerable to a novel "bijection learning" attack that leverages in-context learning to teach the model a custom string-to-string encoding, bypassing built-in safety mechanisms. The attack encodes harmful queries, sends them to the model, and decodes the response, effectively circumventing safety filters. The complexity of the encoding can be controlled, adapting the attack to various LLMs; more capable models are more susceptible to complex encodings.

Large Language Models (LLMs) acting as code assistants may recommend malicious code or resources when presented with prompts framed as programming challenges, even if they refuse similar direct prompts. This occurs due to insufficient context-aware safety mechanisms. LLMs may suggest compromised libraries, malicious APIs, or other attack vectors within seemingly benign code examples.

Large Language Models (LLMs) are vulnerable to transferable ensemble black-box jailbreak attacks. The vulnerability allows an attacker to bypass safety mechanisms and elicit undesired or harmful responses from the LLM by using an ensemble of LLM-as-attacker methods that optimize malicious prompts, adaptively adjusting resources based on prompt difficulty, and strategically modifying prompt semantics to evade detection.

Large Language Models (LLMs) are vulnerable to an attack vector termed "Attack via Implicit Reference" (AIR). AIR bypasses safety mechanisms by decomposing a malicious objective into multiple benign, seemingly unrelated objectives linked through implicit contextual references. The LLM generates harmful content by combining the outputs of these seemingly harmless objectives, without explicitly triggering safety filters designed to detect direct requests for malicious content.

Large Language Models (LLMs) are vulnerable to a novel jailbreak attack that exploits resource limitations. By overloading the model with a computationally intensive preliminary task (e.g., a complex character map lookup and decoding), the attacker prevents the activation of the LLM's safety mechanisms, enabling the generation of unsafe outputs from subsequent prompts. The attack's strength is scalable and adjustable by modifying the complexity of the preliminary task.

Large Language Models (LLMs) are vulnerable to a multi-objective black-box jailbreaking attack (BlackDAN) that optimizes prompts to maximize the likelihood of generating unsafe responses while maintaining contextual relevance and minimizing detectability. The attack leverages a multi-objective evolutionary algorithm (NSGA-II) to balance attack success rate, semantic consistency, and stealthiness, resulting in more effective and less easily detectable jailbreaks than single-objective approaches.

Large Language Models (LLMs) are vulnerable to a multi-turn jailbreak attack, termed "Jigsaw Puzzles" (JSP), which circumvents existing safeguards by splitting harmful questions into harmless fragments. The LLM is prompted to reconstruct and answer the complete question from these fragments, resulting in the generation of harmful responses. The attack relies on the LLM's ability to piece together seemingly benign input to form a malicious query, exploiting the model's contextual understanding…

Large Language Models (LLMs) are vulnerable to multi-turn adversarial attacks where malicious users obscure harmful intents across multiple queries. The ActorAttack method leverages the LLM's own knowledge base to discover semantically linked "actors" related to a harmful target. By posing seemingly innocuous questions about these actors, the attacker guides the LLM towards revealing harmful information step-by-step, accumulating knowledge until the desired malicious output is obtained, even…

A vulnerability in safety-aligned Large Language Models (LLMs) allows attackers to bypass safety mechanisms using adversarial prompt translation. The vulnerability stems from the ability to translate garbled adversarial prompts generated by gradient-based attacks into coherent, human-readable prompts that retain their adversarial capability. This allows for the successful transfer of attacks across different LLMs.

Large language models (LLMs) controlling robots are vulnerable to jailbreaking attacks. The ROBOPAIR algorithm demonstrates that malicious prompts can bypass safety mechanisms, causing robots to perform harmful physical actions. This vulnerability exploits the LLM's reliance on textual prompts and its potential lack of sufficient contextual understanding to prevent unsafe commands. The attack is effective across different access levels.