Prompt layer Vulnerabilities

Large Language Models (LLMs), specifically GPT-3.5-turbo, LLaMA3-8B-instruct, and DeepSeek-R1-Distill-Qwen-7B, are vulnerable to a "Self-Harm" jailbreak attack (Self-HarmLLM). This vulnerability exploits the model's ability to understand its own safety boundaries to generate adversarial inputs against itself. An attacker utilizes a two-session approach: in the first session (Mitigation Session), the attacker instructs the model to rewrite a harmful query into a "Mitigated Harmful Query"…

Large Language Models (LLMs) are vulnerable to Linguistic Style Jailbreaks, a technique where an attacker reframes a harmful prompt using specific linguistic tones—such as politeness, fear, curiosity, or compassion—to bypass safety guardrails. While standard safety alignment (RLHF) effectively filters harmful requests phrased in neutral or hostile tones, it fails to generalize to prompts where the semantic intent remains harmful but the stylistic framing triggers compliant, helpful, or…

Centralized Multi-Agent Systems (MAS) architectures, specifically those implemented using frameworks such as Magentic-One, LangGraph, and OpenAI Agents, contain design-level vulnerabilities facilitating safety alignment bypasses. Two primary mechanisms drive this vulnerability: 1. Context Fragmentation via Atomic Delegation: The orchestrator decomposes adversarial prompts into atomic, granular sub-tasks (e.g., "click button," "write function header") before delegating them to sub-agents. These…

A vulnerability in Large Language Models (LLMs) allows for systematic jailbreaking through a meta-optimization framework called AMIS (Align to MISalign). The attack uses a bi-level optimization process to co-evolve both the jailbreak prompts and the scoring templates used to evaluate them.

The JPRO (Automated Multimodal Jailbreaking via Multi-Agent Collaboration) framework exploits a vulnerability in Large Vision-Language Models (VLMs) related to insufficient cross-modal safety alignment and lack of maliciousness sustainability in multi-turn dialogues. The attack leverages a multi-agent system (Planner, Attacker, Modifier, Verifier) to automate the generation of adversarial image-text pairs. By employing hybrid tactics—such as combining role-playing with malicious content…

Multi-agent Large Language Model (LLM) systems employing ensemble sampling-and-voting strategies (specifically the "Agent Forest" framework) are vulnerable to adversarial input perturbations. While increasing the number of agents ($n \in \{1, \dots, 25\}$) improves accuracy on clean inputs, the system fails to mitigate the impact of synthetic punctuation noise and human-like typographical errors. Attackers can introduce surface-level perturbations—such as random punctuation insertion (10-50%…

A safety bypass vulnerability, dubbed "Ninja" (Needle-in-a-haystack jailbreak), exists in long-context Large Language Models (LLMs). The vulnerability exploits a degradation in safety alignment that occurs when a harmful goal is embedded within a massive, benign context window. Unlike traditional adversarial attacks that use unintelligible strings or "many-shot" attacks that use harmful examples, this method utilizes thematically relevant but innocuous text (the "haystack"). The attack…

Multiple open-weight Large Language Models (LLMs)—specifically those prioritizing capability over safety alignment—exhibit a critical vulnerability to adaptive multi-turn prompt injection and jailbreak attacks. While these models effectively reject isolated, single-turn adversarial inputs (averaging ~13.11% Attack Success Rate), they fail to maintain safety guardrails and policy enforcement across extended conversational contexts. By leveraging iterative strategies such as "Crescendo" (gradual…

The KG-DF (Knowledge Graph Defense Framework) contains a logic vulnerability in its Semantic Parsing Module, specifically within the keyword extraction phase defined as $K_{core} = \text{LLM}(P_{prompt})$. The framework relies on a Large Language Model (e.g., GPT-3.5-turbo) to distill user input into keywords ($K_{core}$), which are then embedded to retrieve security warning triples ($T_{match}$) from a Knowledge Graph.

Multimodal Large Language Models (MLLMs) capable of processing speech and audio are vulnerable to Speech-Audio Compositional Attacks. This vulnerability exists because current safety mechanisms often rely on text-only transcription or fail to analyze the full acoustic context of an input. By manipulating the composition of audio signals, an attacker can bypass safety filters and elicit harmful responses. The attacks exploit three specific mechanisms: (1) Speech Overlap, where harmful…