Language Model Security Database

Large Language Models (LLMs), including Llama-3, Falcon-3, Gemma-2, and GPT-4 variants, are susceptible to system prompt extraction attacks. The vulnerability exists due to the models' instruction-following nature, which allows remote attackers to bypass safety guardrails and retrieve the model's hidden system configuration (system prompt) verbatim. This is successfully exploited using an "Extended Sandwich Attack," where an adversarial extraction command is embedded between benign questions…

Large Language Models (LLMs), specifically Llama-2-7B-Chat and Qwen2.5-Instruct (1.5B and 3B), contain a vulnerability in their post-training safety alignment mechanisms identified as "Refusal Direction Abliteration." The safety alignment in these models creates distinct, isolated neural pathways (a single latent direction in the residual stream) responsible for refusal behavior. An attacker can identify this specific direction by computing the difference in mean activations between harmful…

The LARGO attack exploits a vulnerability in Large Language Models (LLMs) allowing attackers to bypass safety mechanisms through the generation of "stealthy" adversarial prompts. The attack leverages gradient optimization in the LLM's continuous latent space to craft seemingly innocuous natural language suffixes which, when appended to harmful prompts, elicit unsafe responses. The vulnerability stems from the LLM's inability to reliably distinguish between benign and maliciously crafted latent…

Large Language Models (LLMs) employing safety mechanisms based on token-level distribution analysis are vulnerable to a jailbreak attack exploiting distributional discrepancies between alignment data and formally expressed logical statements. The vulnerability allows malicious actors to bypass safety restrictions by translating harmful natural language prompts into equivalent first-order logic expressions. The LLM, trained primarily on natural language, fails to recognize the harmful intent…

Multilingual prompt injection vulnerability in four closed-source Large Language Models (LLMs): GPT-4o, DeepSeek-R1, Gemini-1.5-Pro, and Qwen-Max. Attackers can bypass safety restrictions and elicit harmful or disallowed content by crafting prompts in English or Chinese, leveraging specific structural techniques (e.g., "Two Sides" prompting) that exploit inconsistencies in the models' safety alignment across languages and prompt formats.

Mobile LLM agents utilizing vision-based screen perception (OCR or Multimodal Large Language Models) are vulnerable to Visual Prompt Injection via malicious GUI overlays. An attacker holding the SYSTEM_ALERT_WINDOW permission can deploy non-focusable floating windows (using FLAG_NOT_FOCUSABLE) containing adversarial text or fabricated UI elements over legitimate applications. Because the agent captures the entire screen buffer to interpret the device state, it ingests the adversarial overlay…

Large Vision-Language Models (LVLMs) that utilize a projection layer (adapter) to bridge a vision encoder and a Large Language Model (LLM) contain a vulnerability stemming from the "Modality Gap"—a distributional distance between image and text token embeddings. This gap allows the visual modality to bypass the safety alignment (RLHF/instruction tuning) of the backbone LLM. Attackers can trigger harmful, toxic, or illegal responses to queries that would be refused in text-only contexts by…

Sparse Autoencoders (SAEs), utilized for interpreting the internal residual stream activations of Large Language Models (LLMs) into human-understandable concepts, are vulnerable to adversarial input perturbations. By employing gradient-based optimization techniques adapted for SAEs (specifically a generalized Greedy Coordinate Gradient), an attacker can craft inputs via suffix appending or token replacement that manipulate the SAE's latent feature activations. This vulnerability allows for the…

An adversarial audio perturbation vulnerability exists in open-source Speech Language Models (SLMs), specifically Qwen2-Audio-7B-Instruct and LLaMa-Omni. The vulnerability allows remote attackers to bypass safety alignment mechanisms and jailbreak the model by injecting imperceptible adversarial noise into audio prompts. By utilizing white-box access and Projected Gradient Descent (PGD) optimization, an attacker can manipulate the continuous speech signal to trigger harmful responses (e.g…

A security vulnerability exists in the quantization process of Small Language Models (SLMs) intended for on-device deployment. When full-precision models are compressed using quantization techniques (reducing weights and activations to 4-bit or 8-bit precision), the safety alignment and refusal mechanisms inherent in the original models are degraded or bypassed. This "Quantization-induced Risk" allows the quantized versions of models to respond to harmful, unethical, or illegal queries…