Language Model Security Database

AutoArgue, an LLM-based evaluation framework for Retrieval-Augmented Generation (RAG) systems, is susceptible to evaluation subversion attacks due to the public availability of its judging prompts and reference data structures. An adversarial RAG system (exemplified by the "Crucible" probe) can incorporate "insider knowledge" of the evaluation logic directly into its generation pipeline. By wrapping the generation process with the evaluator's specific prompts, the system can pre-filter…

The STEP-LLM framework, utilized for generating Computer-Aided Design (CAD) STEP files (ISO 10303) from natural language, exhibits a safety alignment vulnerability during the fine-tuning of base Large Language Models (specifically Llama-3.2-3B-Instruct and Qwen-2.5-3B). The training pipeline employs Depth-First Search (DFS) reserialization and Reinforcement Learning (RL) with Scaled Chamfer Distance rewards to optimize for geometric fidelity and syntactic validity of Boundary Representation…

A vulnerability exists in frontier Large Language Models (LLMs) where in-context information (e.g., provided via Retrieval-Augmented Generation) completely overrides parametric safety guardrails when processing counterfactual or adversarial medical evidence. When a prompt contains fabricated clinical context asserting the medical efficacy of toxic substances, illicit drugs, or nonsensical items, the LLM suppresses its internal knowledge of the substance's toxicity. Internal representation…

Web-facing Retrieval-Augmented Generation (RAG) systems are vulnerable to Indirect Prompt Injection (IPI) and retrieval poisoning via web-native markup and Unicode carriers. Standard ingestion pipelines often parse untrusted web pages without stripping invisible constructs, such as hidden HTML spans, off-screen CSS, alt text, ARIA attributes, and zero-width characters. When an attacker embeds malicious instructions within these invisible carriers on third-party sites, the RAG system retrieves…

A vulnerability exists in the function-calling mechanisms of open-source Large Language Models (LLMs), specifically identified as "Renaming Tool Poisoning" (RTP). This attack vector exploits the model's visibility into both the natural language description and the actual code implementation of available tools. By embedding a two-part adversarial payload—one in the tool description directing focus to implementation variables, and another within the tool's source code variable…

A vulnerability exists in the task-planning and execution logic of Large Language Model (LLM) agents, specifically within trip-planning and web-use agents. The vulnerability, identified as a "User-Mediated Attack," occurs because agents prioritize task completion and "helpfulness" over safety verification when processing content provided by the user. When a benign user forwards untrusted external content (e.g., promotional text containing phishing links or malicious instructions) to the agent…

Large Language Model (LLM) agents implementing the Model Context Protocol (MCP) are vulnerable to Implicit Tool Poisoning (ITP). This vulnerability allows an attacker to manipulate agent behavior by embedding malicious instructions within the metadata (specifically the natural language description) of a third-party tool. Unlike explicit tool poisoning, where the agent is tricked into invoking a malicious tool, ITP exploits the agent's contextual reasoning to force the invocation of a distinct…

State-of-the-art secure code generation methods (Sven, SafeCoder, and PromSec) are vulnerable to adversarial prompt perturbations during inference, allowing for the bypass of security alignment mechanisms. The vulnerability stems from the models' reliance on surface-level textual pattern matching rather than semantic security reasoning. By employing simple prompt manipulations—such as Cue Inversion (flipping security directives), Naturalness Reframing (rewriting comments as novice questions)…

LLM-as-a-Reviewer systems, which utilize large language models to automate the peer review process, are vulnerable to the Paraphrasing Adversarial Attack (PAA). PAA is a black-box optimization technique that exploits the model's sensitivity to specific input sequences and self-preference bias. By iteratively paraphrasing specific manuscript sections (such as the abstract) using in-context learning (ICL) guided by previous review scores, an attacker can generate adversarial sequences that…

The paper describes a reproducible black-box indirect prompt injection evaluation for embedding-based RAG and agent systems. It separates a poisoned document into a retrieval-optimized trigger fragment and an instruction-bearing attack fragment, showing that one injected item can be surfaced by natural queries and then influence model output or agent behavior. These are paper-reported findings; they were not independently verified here.