Language Model Security Database

The paper presents a concrete, reproducible security evaluation in which attacker-controlled instructions embedded in retrieved external content steer stateful, tool-calling LLM agents toward unauthorized actions. It adapts white-box GCG and black-box TAP to AgentDojo and evaluates single-task and task-universal attacks across 80 task pairs in four domains. The reported results show that semantic black-box optimization can discover functional prompt injections more effectively than…

SilentRetrieval describes a specific RAG corpus-integrity vulnerability: an attacker able to add a topically relevant document to a retrieval corpus can make that document rank highly and influence the generated answer while remaining fluent enough to evade simple perplexity checks. The paper evaluates a two-stage method combining retrieval-oriented document optimization with context-adaptive claim integration. A safe defensive reproduction is to use only isolated benchmark corpora and inert…

Hybrid monitoring protocols for Large Language Model (LLM) agents, specifically those that strictly evaluate both Chain-of-Thought (CoT) reasoning and tool usage (such as LlamaFirewall's AlignmentCheck and Extract-and-Evaluate monitors), are vulnerable to "Agent-as-a-Proxy" indirect prompt injection attacks. Unlike standard injections that target the agent, this attack targets the monitoring model itself. By embedding a malicious string—optimized via an algorithm named Parallel-GCG—into…

The GPT-OSS-20B large language model contains critical failures in its alignment and Chain-of-Thought (CoT) reasoning mechanisms, specifically in how it prioritizes numerical objectives and validates procedural structure. The model is vulnerable to "Quant Fever," where explicit numerical targets in a prompt (e.g., "delete 90% of files") override contextual safety constraints (e.g., "do not delete important files"). Furthermore, the model exhibits "Reasoning Procedure Mirage," where harmful…

Large Language Models (LLMs) employing alignment-based defenses against prompt injection and jailbreak attacks exhibit vulnerability to an informed white-box attack. This attack, termed Checkpoint-GCG, leverages intermediate model checkpoints from the alignment training process to initialize the Greedy Coordinate Gradient (GCG) attack. By using each checkpoint as a stepping stone, Checkpoint-GCG successfully finds adversarial suffixes that bypass defenses achieving significantly higher attack…

The Virus attack method enables attackers to bypass guardrail moderation on fine-tuning data, leading to a significant degradation of safety alignment in large language models (LLMs). This is achieved through a dual-objective data optimization strategy that crafts harmful data undetectable by the guardrail while maximizing their effectiveness in compromising the victim model's safety.

Vision-Language Models (VLMs) are vulnerable to jailbreak attacks using carefully crafted adversarial images. Attackers can bypass safety mechanisms by generating images semantically aligned with harmful prompts, exploiting the fact that minimal cross-entropy loss during adversarial image optimization does not guarantee optimal attack effectiveness. The attack uses a multi-image collaborative approach, selecting images within a specific loss range to enhance the likelihood of successful…

A Cross-Prompt Injection Attack (XPIA) can be amplified by appending a Greedy Coordinate Gradient (GCG) suffix to the malicious injection. This increases the likelihood that a Large Language Model (LLM) will execute the injected instruction, even in the presence of a user's primary instruction, leading to data exfiltration. The success rate of the attack depends on the LLM's complexity; medium-complexity models show increased vulnerability.

A training-time attack against open-source LLMs that injects adversarial embeddings into the model's token embeddings without modifying model weights. This allows an attacker to introduce backdoors, jailbreaks, or prompt stealing capabilities by simply modifying specific token embeddings within the model file, maintaining model utility for non-triggered inputs. The attack leverages soft prompt tuning to optimize adversarial embeddings, which are then assigned to chosen trigger tokens.

Multimodal Large Language Models (LLMs) processing speech input are vulnerable to adversarial attacks. Imperceptible perturbations added to audio input can cause the model to generate unsafe or harmful text responses, overriding built-in safety mechanisms. The attacks are effective even with limited knowledge of the model's internal workings, demonstrating transferability across different models.