Language Model Security Database

Large Language Models (LLMs) are vulnerable to system instruction leakage when extraction requests are framed as benign formatting, encoding, or structured-output tasks. While standard alignment and refusal mechanisms successfully block direct queries for system instructions, they fail when attackers request the instructions to be rendered in alternate representations (e.g., YAML, TOML, Base64, or system logs). The model's safety filters misinterpret the request as a harmless transformation or…

A vulnerability in LLM-based Multi-Agent Systems (LLM-MAS) allows an attacker who controls a single arbitrary agent to map and extract the system's entire confidential communication topology. Unlike prior attacks that rely on direct identity queries and administrative privileges, this attack infers topology stealthily purely from contextual and linguistic signals (stylometry, role-specific syntax), bypassing standard keyword-based and identity-filtering defenses. The exploit relies on a…

Architectural limitations in Meta's Llama-Prompt-Guard-2-86M and Llama-Guard-3-8B cause them to fail at detecting indirect prompt injections and agentic tool-use attacks, with detection rates dropping as low as 7-37%. Llama-Guard-3-8B enforces strict user/assistant message alternation and lacks support for tool-use roles; attempting to process messages with role: "tool" or role: "ipython" causes the chat template to raise an error, preventing evaluation entirely. PromptGuard 2 operates…

Audio Large Language Models (ALLMs) integrated into voice agent systems for high-stakes domains (banking, IT support, logistics) are vulnerable to multimodal adversarial attacks via spoken interaction. Adversaries can exploit the model's inherent compliance and contextual awareness through multi-turn dialogue to bypass authentication safeguards, escalate privileges (e.g., unauthorized credit limit increases), exfiltrate sensitive Personally Identifiable Information (PII), and poison…

A vulnerability exists in Large Language Model (LLM) deployments and multi-agent systems where an autonomous attacker agent can systematically extract hidden system prompts through self-evolving interaction strategies. The vulnerability leverages a "JustAsk" framework which utilizes Upper Confidence Bound (UCB) exploration to dynamically select and refine attack vectors from a hierarchical taxonomy of 14 atomic skills (e.g., structural formatting, authority appeals) and 14 multi-turn…

Production Large Language Models (LLMs) are vulnerable to long-form training data extraction via a two-phase prompt injection attack. This vulnerability allows an attacker to recover substantial portions of memorized, copyrighted text (such as novels) by exploiting the model's autoregressive text completion capabilities. The attack methodology involves two distinct phases: 1. Prefix Completion Probe: The attacker provides a short "seed" sequence (e.g., the first sentence of a book) coupled…

Large Language Model (LLM) agents operating in tool-augmented environments are susceptible to "Contextual Fragility" and multi-turn "long-chain" exploitation. Existing safety mechanisms predominantly function on a stateless, atomic paradigm, evaluating individual input-output pairs in isolation. This allows an adversary to orchestrate complex attack trajectories where malicious intent is distributed across multiple, individually benign steps (a "Domino Effect"). Consequently, an attacker can…

Multiple open-weight Large Language Models (LLMs)—specifically those prioritizing capability over safety alignment—exhibit a critical vulnerability to adaptive multi-turn prompt injection and jailbreak attacks. While these models effectively reject isolated, single-turn adversarial inputs (averaging ~13.11% Attack Success Rate), they fail to maintain safety guardrails and policy enforcement across extended conversational contexts. By leveraging iterative strategies such as "Crescendo" (gradual…

A vulnerability termed "Controlled-Release Prompting" allows attackers to bypass lightweight input filters (prompt guards) deployed in front of Large Language Models (LLMs). The attack exploits the computational resource asymmetry between the resource-constrained guard model and the highly capable target model. Attackers encode malicious instructions using obfuscation techniques—such as substitution ciphers (Timed-Release) or verbose character descriptions (Spaced-Release)—that require…

DNA language models, such as the Evo series, are vulnerable to jailbreak attacks that coerce the generation of DNA sequences with high homology to known human pathogens. The GeneBreaker framework demonstrates this by using a combination of carefully crafted prompts leveraging high-homology non-pathogenic sequences and a beam search guided by pathogenicity prediction models (e.g., PathoLM) and log-probability heuristics. This allows bypassing safety mechanisms and generating sequences exceeding…