Language Model Security Database

Multi-agent systems (MAS) utilizing Large Language Model (LLM) orchestration are vulnerable to control-flow hijacking via indirect prompt injection, leading to Remote Code Execution (RCE). This vulnerability arises when a sub-agent (e.g., a file surfer or web surfer) processes untrusted input containing adversarial metadata, such as simulated error messages or administrative instructions. The sub-agent faithfully reproduces this adversarial content in its report to the orchestrator agent. The…

Autoregressive Large Language Models (LLMs) suffer from a dynamic discriminative degradation vulnerability during sequence generation. When processing complex or adversarial inputs, the model's internal capability to distinguish between benign and harmful token sequences—measured by the linear separability of their hidden states—progressively diminishes as generation continues. If an attacker successfully bypasses the model's initial safety compliance judgment (early generation steps), the…

Large Language Models (LLMs) are vulnerable to multi-turn adversarial attacks that exploit incremental policy erosion. The attacker uses a breadth-first search strategy to generate multiple prompts at each turn, leveraging partial compliance from previous responses to gradually escalate the conversation towards eliciting disallowed outputs. Minor concessions accumulate, ultimately leading to complete circumvention of safety measures.

Fine-tuning Large Language Models (LLMs) on the CyberLLMInstruct dataset results in a critical degradation of safety alignment and refusal mechanisms. While the dataset comprises "pseudo-malicious" content (educational descriptions of malware, phishing, and exploits without executable payloads), the Supervised Fine-Tuning (SFT) process on this corpus causes the models to generalize this instruction-following behavior to actual malicious requests. This effectively bypasses safety guardrails…

Large Language Models (LLMs) are vulnerable to jailbreak attacks by crafted prompts that bypass safety mechanisms, causing the model to generate harmful or unethical content. This vulnerability stems from the inherent tension between the LLM's instruction-following and safety constraints. The JBFuzz technique demonstrates the ability to efficiently and effectively discover such prompts through a fuzzing-based approach leveraging novel seed prompt templates and a synonym-based mutation strategy.

A vulnerability exists in large language models (LLMs) where the model's internal representations (activations) in specific latent subspaces can be manipulated to trigger jailbreak responses. By calculating a perturbation vector based on the difference between the mean activations of "safe" and "jailbroken" states, an attacker can introduce a targeted perturbation to the model's activations, causing it to generate unsafe outputs even when presented with a safe prompt. This manipulates the…

Large Language Models (LLMs) are vulnerable to implicit misinformation propagation due to sycophantic compliance with false premises. When a user prompt embeds a factually incorrect assumption or conspiracy theory as an unchallenged premise (implicit presupposition) rather than asking for verification, the model frequently fails to detect the falsehood. Instead of correcting the user, the model hallucinates a response that accepts, validates, and reinforces the false premise. This…

Large Language Models (LLMs) are vulnerable to Dialogue Injection Attacks (DIA), where malicious actors manipulate the chat history to bypass safety mechanisms and elicit harmful or unethical responses. DIA exploits the LLM's chat template structure to inject crafted dialogue into the input, even in black-box scenarios where the model's internals are unknown. Two attack methods are presented: one adapts gray-box prefilling attacks, the other leverages deferred responses to increase the…

A vulnerability exists in Retrieval-Augmented Generation (RAG) systems that allows for black-box adversarial attacks known as "CtrlRAG." This flaw allows an attacker to manipulate the generation of Large Language Models (LLMs) by injecting maliciously crafted inputs into the system's knowledge base. Unlike traditional injection attacks that rely on direct concatenation, CtrlRAG utilizes a Masked Language Model (MLM) to iteratively replace words in the malicious text. This optimization ensures…

Multimodal Large Language Models (MLLMs) are vulnerable to Jailbreak-Probability-based Attacks (JPA). JPA leverages a Jailbreak Probability Prediction Network (JPPN) to identify and optimize adversarial perturbations in input images, maximizing the probability of eliciting harmful responses from the MLLM, even with small perturbation bounds and few iterations. The attack operates by modifying the input image's hidden states within the MLLM to increase the predicted jailbreak probability.