Language Model Security Database

Large Language Models (LLMs) utilizing Chain-of-Thought (CoT) prompting are vulnerable to input perturbations that decouple intermediate reasoning from the final answer. An attacker can generate adversarial examples using gradient-based optimization (targeting specific loss functions that maximize reasoning divergence while minimizing answer loss) to induce "Right Answer, Wrong Reasoning" behaviors. This vulnerability manifests through two primary attack vectors: 1. Token-level perturbations…

Mobile LLM agents utilizing vision-based screen perception (OCR or Multimodal Large Language Models) are vulnerable to Visual Prompt Injection via malicious GUI overlays. An attacker holding the SYSTEM_ALERT_WINDOW permission can deploy non-focusable floating windows (using FLAG_NOT_FOCUSABLE) containing adversarial text or fabricated UI elements over legitimate applications. Because the agent captures the entire screen buffer to interpret the device state, it ingests the adversarial overlay…

Sparse Autoencoders (SAEs), utilized for interpreting the internal residual stream activations of Large Language Models (LLMs) into human-understandable concepts, are vulnerable to adversarial input perturbations. By employing gradient-based optimization techniques adapted for SAEs (specifically a generalized Greedy Coordinate Gradient), an attacker can craft inputs via suffix appending or token replacement that manipulate the SAE's latent feature activations. This vulnerability allows for the…

Large Language Models (LLMs) are vulnerable to a novel jailbreak attack, termed ICE (Intent Concealment and Diversion), which leverages hierarchical prompt decomposition and semantic expansion to bypass safety filters. ICE achieves high attack success rates with single queries, exploiting the models' limitations in multi-step reasoning.

A steganographic jailbreak attack, termed StegoAttack, allows bypassing safety mechanisms in Large Language Models (LLMs) by embedding malicious queries within benign-appearing text. The attack hides the malicious query in the first word of each sentence of a seemingly innocuous paragraph, leveraging the LLM's autoregressive generation to process and respond to the hidden query, even when employing encryption in the response.

Large Language Models (LLMs) are vulnerable to robust jailbreak prompts generated by the ArrAttack framework. ArrAttack uses a two-stage process: a robustness judgment model trained to identify prompts that bypass existing LLM safety mechanisms, and a robust jailbreak prompt generation model that leverages this information to create highly effective attacks. This allows attackers to bypass multiple defense mechanisms, including perplexity-based detection, input preprocessing, and…

A vulnerability exists in Vision-Language Models (VLLMs) that allows for transferable, targeted adversarial attacks. Attackers can generate adversarial image perturbations using an ensemble of open-source surrogate models (primarily CLIP-based visual encoders) which effectively transfer to proprietary, black-box VLLMs. The attack leverages a specific optimization framework that combines a Visual Contrastive Loss with multiple positive/negative visual examples, rather than relying solely on…

The Vision-Language Model (VLM) perception module in Vision-and-Language Navigation (VLN) agents is vulnerable to adversarial 3D object injection via the Adversarial Object Fusion (AdvOF) framework. An attacker can generate physically plausible 3D objects with adversarial perturbations capable of deceiving the agent's VLM across multiple viewing angles and distances. The vulnerability exists due to a misalignment between 3D physical manipulations and the agent's 2D image perception, combined…

A vulnerability in several Large Language Models (LLMs) allows bypassing safety mechanisms through targeted noise injection. Explainable AI (XAI) techniques reveal specific layers within the LLM architecture most responsible for content filtering. Injecting noise into these layers or preceding layers circumvents safety restrictions, enabling the generation of harmful or previously prohibited outputs.

A vulnerability exists in the tool selection mechanism of Large Language Model (LLM) agents that utilize a retrieval-then-selection pipeline (RAG) for identifying executable tools. The vulnerability, known as "ToolHijacker," allows a remote attacker to manipulate the agent's decision-making process by injecting a malicious tool document into the accessible tool library (e.g., via third-party tool hubs or plugins). The attack employs a two-phase optimization strategy to craft a malicious tool…