Impact

Reliability Research

Issues affecting model consistency and dependability

203 matching entries

Reliability research results

203 entries

Misleading Option Injection

Published Jan 19, 2026Analyzed Mar 8, 2026Source: arXiv

Large Language Models (LLMs) deployed using Multiple-Choice Question Answering (MCQA) interfaces or choice-based selection structures are vulnerable to Option Injection. By appending a task-irrelevant candidate choice (e.g., Option E) containing a steering directive—specifically utilizing threat framing (penalty coercion) or bonus framing (reward inducement)—an attacker can hijack the model's decision-making process. The vulnerability stems from a flaw in attention allocation: the model's…

OI-Bench: An Option Injection Benchmark for Evaluating LLM Susceptibility to Directive Interference

Evaluated models: GPT-5, GPT-5 Mini, Claude Haiku 4.5 +9 more

Nugget Overfitting Attack

Published Jan 19, 2026Analyzed Feb 22, 2026Source: arXiv

AutoArgue, an LLM-based evaluation framework for Retrieval-Augmented Generation (RAG) systems, is susceptible to evaluation subversion attacks due to the public availability of its judging prompts and reference data structures. An adversarial RAG system (exemplified by the "Crucible" probe) can incorporate "insider knowledge" of the evaluation logic directly into its generation pipeline. By wrapping the generation process with the evaluator's specific prompts, the system can pre-filter…

Insider Knowledge: How Much Can RAG Systems Gain from Evaluation Secrets?

Evaluated models: GPT-4o, Llama 3.3 70B

STEP Manufacturing Corruption

Published Jan 19, 2026Analyzed Feb 22, 2026Source: arXiv

The STEP-LLM framework, utilized for generating Computer-Aided Design (CAD) STEP files (ISO 10303) from natural language, exhibits a safety alignment vulnerability during the fine-tuning of base Large Language Models (specifically Llama-3.2-3B-Instruct and Qwen-2.5-3B). The training pipeline employs Depth-First Search (DFS) reserialization and Reinforcement Learning (RL) with Scaled Chamfer Distance rewards to optimize for geometric fidelity and syntactic validity of Boundary Representation…

STEP-LLM: Generating CAD STEP Models from Natural Language with Large Language Models

Evaluated models: GPT-4o, Llama 3.2 3B, Qwen 2.5 3B

VLM In-the-Loop Adversary

Published Jan 19, 2026Analyzed Mar 9, 2026Source: arXiv

The VILTA (VLM-in-the-Loop Trajectory Adversary) framework is vulnerable to Prompt Injection and Data Poisoning via un-sanitized scene representation inputs. The system integrates a Vision-Language Model (Gemini-2.5-Flash) into a closed-loop reinforcement learning environment, feeding it Bird’s-Eye-View (BEV) imagery alongside text-based vehicle dynamics data (e.g., position, speed, and risk_category) to generate challenging driving trajectories. An attacker who can manipulate the input…

VILTA: A VLM-in-the-Loop Adversary for Enhancing Driving Policy Robustness

Evaluated models: Gemini 2.5 Flash

Stealthy Tool Chain Amplification

Published Jan 16, 2026Analyzed Apr 11, 2026Source: arXiv

A stealthy resource exhaustion (Economic Denial-of-Service) vulnerability exists in the multi-turn tool-calling layer of Large Language Model (LLM) agents, particularly those utilizing the Model Context Protocol (MCP). An attacker controlling a third-party tool server can manipulate text-visible fields (such as argument descriptions and error messages) to force the LLM into a prolonged, verbose tool-calling loop. By demanding lengthy, non-semantic outputs (e.g., long comma-separated lists) and…

Beyond Max Tokens: Stealthy Resource Amplification via Tool Calling Chains in LLM Agents

Evaluated models: DeepSeek R1 Distill Llama 70B, GLM 4.5 Air, GPT-4o +4 more

LLM False Refusal Bias

Published Jan 13, 2026Analyzed Feb 22, 2026Source: arXiv

Large Language Models (LLMs) exhibit a False Refusal vulnerability during legitimate hate speech detoxification tasks (text style transfer). Safety alignment mechanisms fail to contextually distinguish between a benign instruction to "detoxify" or "rewrite" harmful content and the generation of harmful content itself. This results in a denial of service where the model refuses to process the input. This vulnerability is not uniformly distributed; it is statistically biased to…

Analyzing Bias in False Refusal Behavior of Large Language Models for Hate Speech Detoxification

Evaluated models: GPT-3.5, GPT-4o, Llama 3.1 8B +4 more

Plaintext Output Overflow

Published Jan 13, 2026Analyzed Feb 22, 2026Source: arXiv

Large Language Models (LLMs) contain a resource consumption vulnerability termed "Overflow," wherein specific non-adversarial, plain-text prompts trigger excessive text generation that saturates the model's output token budget. This vulnerability exploits the model's alignment towards helpfulness and exhaustiveness, alongside tokenizer inefficiencies (e.g., zero-width characters), to force the generation of maximum-length responses (often exceeding 5,000 tokens) from short inputs. This differs…

BenchOverflow: Measuring Overflow in Large Language Models via Plain-Text Prompts

Evaluated models: GPT-5, Llama 3.1 8B Instruct, Llama 3.2 3B Instruct +5 more

Distractor Inverse Scaling

Published Jan 12, 2026Analyzed Feb 21, 2026Source: arXiv

Reasoning-capable Large Language Models (LLMs) and agentic AI systems exhibit a critical vulnerability to contextual distractors, resulting in catastrophic performance degradation (up to 80% drop in accuracy) and emergent misalignment. When the input context contains noise—specifically random documents, irrelevant chat history, or task-specific "hard negative" distractors—the models fail to filter this information. Instead of ignoring the noise, the models disproportionately attend to…

Lost in the Noise: How Reasoning Models Fail with Contextual Distractors

Evaluated models: Gemini 2.5 Pro, Gemini 2.5 Flash, DeepSeek R1 0528 +4 more

LLM Emoticon Confusion

Published Jan 12, 2026Analyzed Mar 9, 2026Source: arXiv

A vulnerability in Large Language Models (LLMs) and autonomous agent frameworks, termed "Emoticon Semantic Confusion," allows for the generation and execution of unintended, potentially destructive code. Because ASCII-based emoticons (e.g., ~, *, !(^^)!) heavily overlap with the symbol space of programming operators, shell wildcards, and file paths, LLMs frequently misinterpret these affective, non-verbal cues as executable directives. When processing user instructions in code-generation or…

False Friends in the Shell: Unveiling the Emoticon Semantic Confusion in Large Language Models

Evaluated models: Claude Haiku 4.5, Gemini 2.5 Flash, GPT-4.1 Mini +3 more

Strategic Abstention Failure

Published Jan 12, 2026Analyzed Feb 21, 2026Source: arXiv

A "risk-invariance" vulnerability exists in Large Language Models (LLMs) wherein the model's decision-making policy is functionally decoupled from its verbalized confidence and externally defined error penalties. Despite generating calibrated confidence estimates (internal epistemic uncertainty), affected models fail to adjust their abstention thresholds when presented with high-penalty incentives (e.g., negative utility for incorrect answers). This results in "utility collapse," where models…

Are LLM Decisions Faithful to Verbal Confidence?

Evaluated models: GPT-5 Mini, GPT-5 Nano, GPT-4.1 Mini +7 more

Related research categories