Attack Context

API Research

Security research involving model API implementations

83 matching entries

API research results

83 entries

Universal Steering Jailbreak

Published 9/1/2025

Analyzed 12/9/2025

A vulnerability exists in the alignment mechanisms of Large Language Models (LLMs) where activation steering—the process of injecting vectors into hidden states during inference—can systematically bypass refusal safeguards. By modifying the residual stream activations at intermediate layers (typically $\lfloor L/2 \rfloor$) using the formula $\overline{\mathbf{x}}_{i}^{(l)}=\mathbf{x}_{i}^{(l)}+\alpha\mathbf{v}$, an attacker can force the model to comply with harmful requests. Research…

The Rogue Scalpel: Activation Steering Compromises LLM Safety

Evaluated models: Llama 3 8B, Llama 3.1 8B, Qwen 2.5 7B +1 more

Source: arXiv

Agent Tool Metadata Lure

Published 8/1/2025

Analyzed 2/22/2026

A vulnerability exists in the tool selection mechanisms of Large Language Model (LLM) agents, identified as the "Attractive Metadata Attack" (AMA). This flaw allows an adversary to manipulate the metadata (names, descriptions, and parameter schemas) of malicious external tools to statistically maximize the likelihood of their selection by the agent, without requiring prompt injection or access to model internals. The vulnerability exploits the agent’s semantic scoring function used to map user…

Attractive Metadata Attack: Inducing LLM Agents to Invoke Malicious Tools

Evaluated models: GPT-4o Mini, Llama 3.3 70B Instruct, Qwen 2.5 32B Instruct +2 more

Source: arXiv

KV-Cache Sharing Timing Side-channel

Published 8/1/2025

Analyzed 1/14/2026

Multi-tenant Large Language Model (LLM) inference systems utilizing global Key-Value (KV) cache sharing are vulnerable to a timing side-channel attack. By measuring the Time-To-First-Token (TTFT) latency of crafted API requests, an unprivileged remote attacker can determine if specific token sequences have been previously processed and cached by the system for other users. This observable timing difference between cache hits (low TTFT) and cache misses (high TTFT) allows for the token-by-token…

Selective KV-Cache Sharing to Mitigate Timing Side-Channels in LLM Inference

Evaluated models: Phi-4 14B, Qwen 3 30B-A3B, Qwen 3 32B +3 more

Source: arXiv

MDH: Hybrid Jailbreak Detection Strategy

Published 8/1/2025

Analyzed 8/31/2025

Large language models that support a developer role in their API are vulnerable to a jailbreaking attack that leverages malicious developer messages. An attacker can craft a developer message that overrides the model's safety alignment by setting a permissive persona, providing explicit instructions to bypass refusals, and using few-shot examples of harmful query-response pairs. This technique, named D-Attack, is effective on its own. A more advanced variant, DH-CoT, enhances the attack by…

Jailbreaking Commercial Black-Box LLMs with Explicitly Harmful Prompts

Evaluated models: GPT-3.5 Turbo, GPT-4o, GPT-4.1 +13 more

Source: arXiv

Forged Assistant Message Jailbreak

Published 7/1/2025

Analyzed 1/14/2026

A vulnerability termed "Trojan Horse Prompting" exists in conversational multimodal models, specifically demonstrated on Google’s Gemini-2.0-flash-preview-image-generation. The vulnerability allows an attacker to bypass safety alignment mechanisms (RLHF and SFT) by manipulating the structural protocol of the conversational API. Unlike standard jailbreaks that manipulate the user prompt, this attack exploits "Asymmetric Safety Alignment" by forging a conversational history where the role is…

Trojan Horse Prompting: Jailbreaking Conversational Multimodal Models by Forging Assistant Message

Evaluated models: Gemini 2.0 Flash Preview Image Generation

Source: arXiv

LLM Interpreter Resource Exhaustion

Published 7/1/2025

Analyzed 8/16/2025

Large Language Models (LLMs) equipped with native code interpreters are vulnerable to Denial of Service (DoS) via resource exhaustion. An attacker can craft a single prompt that causes the interpreter to execute code that depletes CPU, memory, or disk resources. The vulnerability is particularly pronounced when a resource-intensive task is framed within a plausibly benign or socially-engineered context ("indirect prompts"), which significantly lowers the model's likelihood of refusal compared…

Running in CIRCLE? A Simple Benchmark for LLM Code Interpreter Security

Evaluated models: Gemini 2.0 Flash, Gemini 2.5 Flash, Gemini 2.5 Pro +5 more

Source: arXiv

Agent API Goal Divergence

Published 6/1/2025

Analyzed 12/9/2025

Large Language Model (LLM) agents capable of invoking external APIs are vulnerable to intent integrity violations. When an agent receives natural language instructions that are ambiguous, underspecified, or contain values not supported by the underlying API schema, the agent frequently fails to preserve user intent. Instead of rejecting the request or asking for clarification, the model may hallucinate parameter values, map unsupported requests to unsafe defaults, or execute actions on…

TAI3: Testing Agent Integrity in Interpreting User Intent

Evaluated models: GPT-4o Mini, Llama 3.1 8B, Qwen 3 30B-A3B +5 more

Source: arXiv

Fine-Tuning Safety Subspace Erosion

Published 6/1/2025

Analyzed 12/9/2025

Large Language Models (LLMs) aligned via techniques such as Reinforcement Learning with Human Feedback (RLHF) or Direct Preference Optimization (DPO) contain a vulnerability in how safety features are encoded within the model parameters. The safety-critical information is primarily stored in low-rank subspaces of the weight matrices (specifically, the difference between the base and aligned model weights). These low-rank subspaces are highly sensitive to parameter updates. Consequently…

Lox: Low-rank extrapolation robustifies llm safety against fine-tuning

Evaluated models: GPT-3.5, Llama 2 7B, Mistral 7B

Source: arXiv

LLM Judge Subversion

Published 6/1/2025

Analyzed 12/9/2025

Alibaba Cloud PAI-Judge and PAI-Judge-Plus are vulnerable to a composite adversarial attack that exploits attention mechanism limitations in Large Language Models (LLMs). An authenticated attacker can manipulate automated evaluation outcomes by appending a long, irrelevant text suffix (approximately 1000 to 2000+ characters) to a response containing adversarial perturbations. This "long-suffix" strategy overwhelms the judge model's context window, causing the attention mechanism to degrade and…

LLMs Cannot Reliably Judge (Yet?): A Comprehensive Assessment on the Robustness of LLM-as-a-Judge

Evaluated models: GPT-4o, Llama 3.1 8B, Llama 3.3 70B +3 more

Source: arXiv

Variational Jailbreak Inference

Published 6/1/2025

Analyzed 7/14/2025

VERA, a variational inference framework, enables the generation of diverse and fluent adversarial prompts that bypass safety mechanisms in large language models (LLMs). The attacker model, trained through a variational objective, learns a distribution of prompts likely to elicit harmful responses, effectively jailbreaking the target LLM. This allows for the generation of novel attacks that are not based on pre-existing, manually crafted prompts.

VERA: Variational Inference Framework for Jailbreaking Large Language Models

Evaluated models: Baichuan 2 7B, Gemini Pro, GPT-3.5 Turbo +8 more

Source: arXiv

Related research categories