Language Model Security Database

AutoArgue, an LLM-based evaluation framework for Retrieval-Augmented Generation (RAG) systems, is susceptible to evaluation subversion attacks due to the public availability of its judging prompts and reference data structures. An adversarial RAG system (exemplified by the "Crucible" probe) can incorporate "insider knowledge" of the evaluation logic directly into its generation pipeline. By wrapping the generation process with the evaluator's specific prompts, the system can pre-filter…

Neural Ranking Models (NRMs) utilizing Transformer architectures (specifically BERT and T5-based re-rankers) are vulnerable to minimal adversarial perturbations that artificially promote a target document's rank. The vulnerability allows an attacker to manipulate ranking outcomes by inserting or substituting a single "query center" token—a word identified as the semantic centroid of the user's query—into the target document. The attack exploits the model's sensitivity to specific semantic…

Large Language Models (LLMs) integrated with external retrieval mechanisms (e.g., Retrieval-Augmented Generation (RAG), web search, or email processing) are vulnerable to Indirect Prompt Injection. This vulnerability occurs when an LLM consumes input from untrusted external sources—such as websites, code repositories, or incoming emails—that contain embedded adversarial prompts. Unlike direct injection, where the user attacks the model, here the "poisoned" data is retrieved by the system…

Retrieval-Augmented Generation (RAG) systems employing standard dense embedding models (e.g., Sentence-T5, SimCSE-BERT, RoBERTa, MPNet) for End-Cloud collaboration are vulnerable to Embedding Inversion Attacks (EIA). While embeddings are vector representations designed to be human-unrecognizable, they retain sufficient semantic information to allow an attacker with access to the vectors (e.g., a malicious or compromised cloud provider) to reconstruct the original sensitive plaintext input.