Language Model Security Database

Large Language Models (LLMs) are vulnerable to a jailbreak attack termed Paper Summary Attack (PSA). An attacker can bypass safety alignment mechanisms by framing a harmful query within the context of a summarized academic paper. The model's propensity to trust the authoritative structure and tone of a research paper summary overrides its safety filters, leading it to process and respond to the embedded malicious instruction. The vulnerability is particularly potent when using summaries of…

LLM-powered agentic systems that use external tools are vulnerable to prompt injection attacks that cause them to bypass their explicit policy instructions. The vulnerability can be exploited through both direct user interaction and indirect injection, where malicious instructions are embedded in external data sources processed by the agent (e.g., documents, API responses, webpages). These attacks cause agents to perform prohibited actions, leak confidential data, and adopt unauthorized…

Audio-based Large Language Models (ALLMs), specifically Qwen2-Audio, are vulnerable to over-the-air adversarial audio attacks. An attacker with white-box access can generate robust adversarial audio perturbations using gradient-based optimization combined with audio augmentation techniques (specifically SpecAugment, translation, and additive noise). These perturbations, when played through a speaker in the physical environment, manipulate the ALLM processing the audio via a microphone. This…

Retrieval-Augmented Generation (RAG) systems utilizing dense (e.g., BERT-based) or sparse (e.g., BM25) retrievers are vulnerable to black-box adversarial prompt injection attacks. By employing a gradient-free Differential Evolution (DE) optimization algorithm (referred to as DeRAG), an attacker can generate short adversarial suffixes (typically ≤ 5 tokens). When these suffixes are appended to a user query, they manipulate the retriever's ranking mechanism to promote a specific, malicious, or…

Reasoning-capable Large Language Models (LLMs) are vulnerable to a class of indirect prompt injection known as Copy-Guided Attacks (CGA). This vulnerability exploits the intrinsic behavior of reasoning models to copy tokens from the input prompt (such as variable names, function identifiers, or code snippets) into their intermediate reasoning traces (Chain-of-Thought). By embedding adversarial trigger sequences into external payloads—specifically within data the model is expected to analyze—an…

A targeted adversarial attack vulnerability exists in Multimodal Large Language Models (MLLMs) susceptible to Adversarial-Guided Diffusion (AGD). This technique generates adversarial images by injecting targeted semantic information into the noise component of the reverse-diffusion process within a text-to-image generative model (specifically Stable Diffusion). Unlike traditional pixel-based attacks that introduce high-frequency perturbations easily removed by low-pass filtering, AGD utilizes…

Large Language Models (LLMs) employing Verbal Confidence Elicitation (CEM)—where the model outputs a numeric confidence score (e.g., "Confidence: 90%") alongside an answer—are vulnerable to Verbal Confidence Attacks (VCAs). Adversaries can manipulate these confidence scores through two primary vectors: perturbation-based attacks (VCA-TF, VCA-TB, SSR) utilizing synonym substitution, typos, and token removal; and jailbreak-based attacks (ConfidenceTriggers, AutoDAN) utilizing optimized trigger…

A vulnerability exists in Large Language Model (LLM)-based Multi-Agent Systems (MAS) that allows a malicious agent to covertly disrupt collaborative decision-making processes without triggering standard safety filters or anomaly detection. This "intention-hiding" attack occurs when an agent adopts a persona that appears linguistically fluent and role-consistent but strategically steers the group toward incorrect outcomes or resource exhaustion. The attacker leverages specific semantic…

A vulnerability exists in Large Language Diffusion Models (LLDMs) due to their parallel denoising architecture. The PArallel Decoding (PAD) jailbreak attack exploits this architecture by injecting multiple, semantically innocuous "sequence connectors" (e.g., "Step 1:", "First") at distributed locations within the initial masked sequence. During the parallel denoising process, these injected tokens act as anchor points that bias the probability distribution of adjacent token predictions. This…

A vulnerability exists in Large Language Models, including GPT-3.5 and GPT-4, where safety guardrails can be bypassed using Trojanized prompt chains within a simulated educational context. An attacker can establish a benign, pedagogical persona (e.g., a curious student) over a multi-turn dialogue. This initial context is then exploited to escalate the conversation toward requests for harmful or restricted information, which the model provides because the session's context is perceived as safe…