Language Model Security Database

Large Language Models (LLMs) incorporating safety filters are vulnerable to a "Prompt, Divide, and Conquer" attack. This attack segments a malicious prompt into smaller, seemingly benign parts, processes these segments in parallel across multiple LLMs, and then reassembles the results to generate malicious code, bypassing the safety filters. The attack's success relies on the iterative refinement of initially abstract function descriptions into concrete implementations. Individual LLM safety…

Large Language Models (LLMs) are vulnerable to implicit misinformation propagation due to sycophantic compliance with false premises. When a user prompt embeds a factually incorrect assumption or conspiracy theory as an unchallenged premise (implicit presupposition) rather than asking for verification, the model frequently fails to detect the falsehood. Instead of correcting the user, the model hallucinates a response that accepts, validates, and reinforces the false premise. This…

This vulnerability allows attackers to identify the presence and location (input or output stage) of specific guardrails implemented in Large Language Models (LLMs) by using carefully crafted adversarial prompts. The attack, termed AP-Test, leverages a tailored loss function to optimize these prompts, maximizing the likelihood of triggering a specific guardrail while minimizing triggering others. Successful identification provides attackers with valuable information to design more effective…

A vulnerability in Large Language Models (LLMs) allows adversarial reasoning attacks to bypass safety mechanisms and elicit harmful responses. The vulnerability stems from the insufficient robustness of existing LLM safety measures against iterative prompt refinement guided by a loss function that measures the LLM's proximity to generating a target harmful response. This allows an attacker to effectively navigate the prompt space, even against adversarially trained models, resulting in…

Vision-Language Models (VLMs), specifically the LLaVA-1.5 and LLaVA-1.6 series, are vulnerable to optimization-based white-box jailbreak attacks despite standard safety alignment measures like Supervised Fine-Tuning (SFT) and Direct Preference Optimization (DPO). Attackers can craft adversarial perturbations in the image space (imperceptible noise) or latent space using Projected Gradient Descent (PGD) to manipulate the model's internal representations. These perturbations maximize the…

Commercial LLM-powered agents utilizing autonomous web access, memory modules, and retrieval-augmented generation (RAG) are vulnerable to indirect prompt injection and environmental manipulation. Attackers can embed malicious instructions into external data sources trusted by the agent (such as Reddit posts, public databases, or ArXiv papers). When the agent autonomously retrieves and processes this content during task execution, it executes the embedded malicious commands. This vulnerability…

A vulnerability exists in large language models (LLMs) where insufficient sanitization of system prompts allows attackers to extract sensitive information embedded within those prompts. Attackers can use an agentic approach, employing multiple interacting LLMs (as demonstrated in the referenced research), to iteratively refine prompts and elicit confidential data from the target LLM's responses. The vulnerability is exacerbated by the LLM's ability to infer context from seemingly innocuous…

CRI (Compliance Refusal Initialization) initializes jailbreak attacks by leveraging pre-trained jailbreak prompts, effectively guiding the optimization process towards the compliance subspace of harmful prompts. This significantly enhances the success rate and reduces the computational overhead of attacks, often requiring only a single optimization step to bypass safety mechanisms. Attacks utilizing CRI demonstrate significantly improved ASR (Adversarial Success Rate) and reduced median steps…

The Knowledge-Distilled Attacker (KDA) model, when used to generate prompts for large language models (LLMs), can bypass LLM safety mechanisms resulting in the generation of harmful, inappropriate, or misaligned content. KDA's effectiveness stems from its ability to generate diverse and coherent attack prompts efficiently, surpassing existing methods in attack success rate and speed. The vulnerability lies in the LLMs' insufficient defenses against the diverse prompt generation strategies…

FC-Attack leverages automatically generated flowcharts containing step-by-step descriptions derived or rephrased from harmful queries, combined with a benign textual prompt, to jailbreak Large Vision-Language Models (LVLMs). The vulnerability lies in the model's susceptibility to visual prompts containing harmful information within the flowcharts, thus bypassing safety alignment mechanisms.