Attack Context

API Research

Security research involving model API implementations

10 matching entries out of 83 in this category

API research results

10 entries

Long-Horizon Agent Attacks

Published Feb 18, 2026Analyzed Apr 10, 2026Source: arXiv

LLM agents equipped with tool-use, persistent memory, and environmental interaction capabilities are vulnerable to long-horizon attacks. Attackers can bypass single-turn safety guardrails by exploiting the temporal dimension of multi-turn interactions to incrementally steer agent behavior. The vulnerability manifests because the agent's safety mechanisms perform localized, single-step evaluations but fail to maintain semantic safety across extended interaction trajectories. This enables…

AgentLAB: Benchmarking LLM Agents against Long-Horizon Attacks

Evaluated models: GPT-4o, GPT-5.1, Gemini 3 Flash+1 more

GPT-4o
GPT-5.1
Gemini 3 Flash
Claude Sonnet 4.5

Autonomous Agent Prompt Reveal

Published Jan 29, 2026Analyzed Feb 21, 2026Source: arXiv

A vulnerability exists in Large Language Model (LLM) deployments and multi-agent systems where an autonomous attacker agent can systematically extract hidden system prompts through self-evolving interaction strategies. The vulnerability leverages a "JustAsk" framework which utilizes Upper Confidence Bound (UCB) exploration to dynamically select and refine attack vectors from a hierarchical taxonomy of 14 atomic skills (e.g., structural formatting, authority appeals) and 14 multi-turn…

Just Ask: Curious Code Agents Reveal System Prompts in Frontier LLMs

Evaluated models: o1, Llama 3.1 70B Hanami X1, Phi-4+38 more

o1
Llama 3.1 70B Hanami X1
Phi-4
Aion 1.0
Sonar Pro
Command A
Llama 4 Maverick
Llama 3.1 Nemotron Ultra 253B v1
Qwen 3 235B-A22B
Mercury
Hunyuan A13B Instruct
UI-TARS 1.5 7B
GPT-oss 120B
Jamba Mini 1.7
Hermes 4 70B
Step 3
LongCat Flash Chat
Tongyi DeepResearch 30B-A3B
Cydonia 24B v4.1
ERNIE 4.5 21B-A3B Thinking
Granite 4.0 H Micro
LFM2 8B-A1B
Nova Premier v1
Kimi K2 Thinking
KAT Coder Pro
Cogito v2.1 671B
Gemini 3 Pro Preview
Grok 4.1 Fast
Claude Opus 4.5
TNG R1T Chimera
Intellect 3
DeepSeek V3.2 Speciale
Trinity Mini
Mistral Large 2512
DeepSeek V3.1 Nex N1
MiMo V2 Flash
GLM-4.7
MiniMax M2.1
Seed 1.6
Molmo 2 8B
GPT-5.2 Codex

Adaptive Tool-Disguised Jailbreak

Published Jan 9, 2026Analyzed Feb 20, 2026Source: arXiv

Large Language Models (LLMs) supporting function calling (tool use) are vulnerable to a jailbreak attack known as iMIST (interactive Multi-step Progressive Tool-disguised Jailbreak Attack). The vulnerability stems from a disparity in alignment training: while models are heavily aligned to refuse harmful natural language generation, they lack sufficient alignment regarding the generation of harmful content within structured data (JSON) used for tool parameters.

Jailbreaking Large Language Models through Iterative Tool-Disguised Attacks via Reinforcement Learning

Evaluated models: Llama 3.1 8B, DeepSeek-V3 671B

Dual Stego MLLM Jailbreak

Published Dec 23, 2025Analyzed Dec 30, 2025Source: arXiv

Commercial Multimodal Large Language Model (MLLM) integrated systems are vulnerable to a "Dual Steganography" jailbreak paradigm (referred to as Odysseus). The vulnerability arises from the reliance of safety filters on the assumption that malicious content must be explicitly visible in the input or output modalities (text or image). Attackers can bypass these filters by encoding malicious queries into binary matrices and embedding them into benign-looking images using steganographic encoders…

Odysseus: Jailbreaking Commercial Multimodal LLM-integrated Systems via Dual Steganography

Evaluated models: GPT-4o, Gemini 2.0 Pro, Gemini 2.0 Flash+1 more

GPT-4o
Gemini 2.0 Pro
Gemini 2.0 Flash
Grok 3

TeleAI Reveals Systemic LLM Vulnerabilities

Published Dec 5, 2025Analyzed Mar 8, 2026Source: arXiv

Reasoning-specialized Large Language Models (LLMs) that utilize Chain-of-Thought (CoT) processes are vulnerable to reasoning-exploitation jailbreaks. Attackers can bypass standard safety alignments (such as RLHF) by using adaptive multi-turn interactions or semantic transformations to induce the model to generate intermediate reasoning steps that "rationalize" or "contextualize" a harmful request. Because current alignment techniques often fail to scale linearly with reasoning depth, forcing…

TeleAI-Safety: A comprehensive LLM jailbreaking benchmark towards attacks, defenses, and evaluations

Evaluated models: GPT-5, GPT-4.1, GPT-4.1 Mini+11 more

GPT-5
GPT-4.1
GPT-4.1 Mini
GPT-4o Mini
o1
Grok 3
Grok 3 Mini
Claude 3.5 Sonnet
Gemini 2.5 Pro
Vicuna 7B
Llama 3.1 8B Instruct
DeepSeek R1
Qwen 1.5 7B Chat
Qwen 2.5 7B Instruct

Existential Tool Invocation

Published Nov 24, 2025Analyzed Dec 8, 2025Source: arXiv

A safety bypass vulnerability exists in Large Language Models (LLMs) and Multi-Modal Models (LVLMs) that expose a "Prefix Completion" parameter in their inference APIs (e.g., echo or prefix parameters). The vulnerability exploits the model's autoregressive nature, where the model prioritizes local coherence with a user-supplied output prefix over global safety alignment (RLHF) or system instructions. By supplying a prefix that establishes a hostile narrative or intent (e.g., "My actions are as…

Can LLMs Threaten Human Survival? Benchmarking Potential Existential Threats from LLMs via Prefix Completion

Evaluated models: DeepSeek V3, Qwen 2.5 7B

Agent Tool Metadata Lure

Published Aug 4, 2025Analyzed Feb 22, 2026Source: arXiv

A vulnerability exists in the tool selection mechanisms of Large Language Model (LLM) agents, identified as the "Attractive Metadata Attack" (AMA). This flaw allows an adversary to manipulate the metadata (names, descriptions, and parameter schemas) of malicious external tools to statistically maximize the likelihood of their selection by the agent, without requiring prompt injection or access to model internals. The vulnerability exploits the agent’s semantic scoring function used to map user…

Attractive Metadata Attack: Inducing LLM Agents to Invoke Malicious Tools

Evaluated models: GPT-4o Mini, Llama 3.3 70B Instruct, Qwen 2.5 32B Instruct+2 more

GPT-4o Mini
Llama 3.3 70B Instruct
Qwen 2.5 32B Instruct
Gemma 3 27B IT
Qwen 3 32B

Conditional Prompt Hijack

Published May 22, 2025Analyzed Dec 8, 2025Source: arXiv

The SPECTRE framework introduces a black-box adversarial attack vector against Large Language Models (LLMs) that utilizes malicious system prompts to hijack conversations. Unlike traditional jailbreaks that aim to bypass safeguards for all inputs, SPECTRE optimizes system prompts to induce incorrect or harmful responses only for specific targeted questions (e.g., "Are COVID vaccines safe?", "Who should I vote for?"), while maintaining high accuracy and benign behavior on all other non-targeted…

SPECTRE: Conditional System Prompt Poisoning to Hijack LLMs

Evaluated models: GPT-3.5 Turbo, GPT-4o Mini, Llama 2 7B+8 more

GPT-3.5 Turbo
GPT-4o Mini
Llama 2 7B
Llama 2 13B
Llama 3.1 8B
DeepSeek 7B
Qwen 2.5 3B
Qwen 2.5 7B
Qwen 2.5 14B
Qwen 2.5 32B
Pythia 12B

Flowchart-based LVLM Jailbreak Attack

Published Feb 28, 2025Analyzed Mar 19, 2025Source: arXiv

FC-Attack leverages automatically generated flowcharts containing step-by-step descriptions derived or rephrased from harmful queries, combined with a benign textual prompt, to jailbreak Large Vision-Language Models (LVLMs). The vulnerability lies in the model's susceptibility to visual prompts containing harmful information within the flowcharts, thus bypassing safety alignment mechanisms.

FC-Attack: Jailbreaking Large Vision-Language Models via Auto-Generated Flowcharts

Evaluated models: Claude 3.5 Sonnet 20240620, Gemini 1.5 Flash, GPT-4o 2024-08-06+4 more

Claude 3.5 Sonnet 20240620
Gemini 1.5 Flash
GPT-4o 2024-08-06
GPT-4o Mini 2024-07-18
InternVL2.5 8B
LLaVA NeXT 8B
Qwen2-VL 7B Instruct

Guardrail Bypass Harmful Fine-tuning

Published Jan 29, 2025Analyzed Mar 19, 2025Source: arXiv

The Virus attack method enables attackers to bypass guardrail moderation on fine-tuning data, leading to a significant degradation of safety alignment in large language models (LLMs). This is achieved through a dual-objective data optimization strategy that crafts harmful data undetectable by the guardrail while maximizing their effectiveness in compromising the victim model's safety.

Virus: Harmful Fine-tuning Attack for Large Language Models Bypassing Guardrail Moderation

Evaluated models: Llama 3 8B, Llama Guard 2

Related research categories