Attack Type

Hallucination Research

Exploits causing models to generate false information

10 matching entries out of 53 in this category

Hallucination research results

10 entries

LLM Scaling Fidelity Paradox

Published Feb 10, 2026Analyzed Feb 21, 2026Source: arXiv

A vulnerability exists in Large Language Model (LLM) context compression architectures (specifically compressor-decoder setups) characterized as the "Size-Fidelity Paradox." When scaling up the parameter count of the compressor model (e.g., beyond 4B parameters in Qwen-3 and LLaMA-3.2 families), the system exhibits a degradation in faithful preservation of the source text, despite improvements in standard training loss and perplexity metrics. This degradation manifests through two primary…

When Less is More: The LLM Scaling Paradox in Context Compression

Evaluated models: Qwen 3 8B, Qwen 3 32B, Llama 3.2 11B+1 more

Qwen 3 8B
Qwen 3 32B
Llama 3.2 11B
Llama 3.2 90B

Safety Steering Amplifies Jailbreaks

Published Feb 5, 2026Analyzed Feb 21, 2026Source: arXiv

Inference-time intervention techniques (also known as activation steering or model steering), utilized to adjust Large Language Model (LLM) behavior without retraining, contain a vulnerability related to robust specificity. When these methods are applied to reduce "over-refusal" (increasing compliance on benign but sensitive-sounding queries), they inadvertently degrade the model's adversarial robustness. Specifically, steering vectors derived from methods such as Difference-in-Means…

Steering Safely or Off a Cliff? Rethinking Specificity and Robustness in Inference-Time Interventions

Evaluated models: Llama 3.1 8B, Llama 3.2 3B, Qwen 2.5 7B+1 more

Llama 3.1 8B
Llama 3.2 3B
Qwen 2.5 7B
Gemma 2 2B

CoT Causal Bypass

Published Feb 3, 2026Analyzed Mar 9, 2026Source: arXiv

Large language models (LLMs) exhibit a "Causal Bypass" vulnerability during Chain-of-Thought (CoT) prompting, where the generated reasoning text does not causally determine the model's final output. Instead of utilizing the explicit CoT tokens, the model routes decision-critical computation through latent, implicit pathways. This allows the visible reasoning trace to function as an unfaithful, post-hoc rationalization rather than an actual representation of the model's internal logic…

Bypassing the Rationale: Causal Auditing of Implicit Reasoning in Language Models

Evaluated models: Phi-4 Mini Reasoning, Qwen 3 1.7B, Phi-3.5 Mini Instruct+7 more

Phi-4 Mini Reasoning
Qwen 3 1.7B
Phi-3.5 Mini Instruct
Phi-2
Qwen 3 0.6B
DialoGPT Large
Phi-1.5
Phi-4
Qwen 3 8B
Qwen 3 4B

RL Adversarial Function Call

Published Jan 27, 2026Analyzed Feb 22, 2026Source: arXiv

Large Language Models (LLMs) enabled with Function Calling (FC) capabilities are vulnerable to adversarial query rewriting and semantic manipulation. Standard FC models, typically trained via Supervised Fine-Tuning (SFT) on static datasets, fail to generalize against adversarial inputs that deviate from fixed distribution patterns. An attacker can exploit this by crafting queries that are semantically similar to valid requests but engineered to induce "bad cases," such as incorrect tool…

Exploring Weaknesses in Function Call Models via Reinforcement Learning: An Adversarial Data Augmentation Approach

Evaluated models: Qwen 2.5 7B Instruct, Qwen 3 0.6B, Qwen 3 4B+1 more

Qwen 2.5 7B Instruct
Qwen 3 0.6B
Qwen 3 4B
Qwen 3 8B

Sound Induces Trimodal Collapse

Published Jan 20, 2026Analyzed Feb 22, 2026Source: arXiv

A vulnerability exists in trimodal audio-video-language models where an attacker can systematically degrade multimodal reasoning through untargeted, audio-only adversarial perturbations. By optimizing a shared perturbation $\delta$ applied to the audio channel, an attacker can manipulate internal representations—specifically targeting audio encoder embeddings and cross-modal attention mechanisms—without modifying visual or textual inputs. The attack exploits the model's reliance on the audio…

SoundBreak: A Systematic Study of Audio-Only Adversarial Attacks on Trimodal Models

Evaluated models: VideoLLAMA2, Qwen 2 7B Instruct, Whisper Large-v2+1 more

VideoLLAMA2
Qwen 2 7B Instruct
Whisper Large-v2
PANNs

Detector Telemetry Camouflage

Published Jan 19, 2026Analyzed Feb 21, 2026Source: arXiv

Single-pass hallucination detectors relying on internal telemetry (uncertainty, hidden-state geometry, and attention patterns) are vulnerable to white-box, model-side adversarial attacks. An attacker can employ the CORVUS (Camouflaging Open-weight Representations, Volumes, Uncertainty, and Structure) technique to fine-tune lightweight Low-Rank Adapters (LoRA) on the target LLM. This method optimizes a specific loss objective that camouflages detector-visible telemetry signals—specifically…

CORVUS: Red-Teaming Hallucination Detectors via Internal Signal Camouflage in Large Language Models

Evaluated models: Llama 2 7B, Llama 3 8B, Qwen 2.5 14B+1 more

Llama 2 7B
Llama 3 8B
Qwen 2.5 14B
Vicuna 7B

Point Cloud Adversarial Attack

Published Jan 10, 2026Analyzed Feb 22, 2026Source: arXiv

Point-based 3D Vision-Language Models (VLMs), specifically PointLLM and GPT4Point, are vulnerable to white-box, gradient-based adversarial attacks. The vulnerability exists in the model's processing of 3D point cloud data, where an attacker can optimize imperceptible geometric perturbations ($\delta$) on the input point cloud ($x$) to manipulate the model's textual output. The paper identifies two specific attack vectors: 1. Vision Attack: Directly perturbs the high-dimensional visual token…

On the Adversarial Robustness of 3D Large Vision-Language Models

Evaluated models: Vicuna 7B

JailFlip Implicit Harm

Published Jun 9, 2025Analyzed Dec 8, 2025Source: arXiv

A vulnerability exists in the safety alignment mechanisms of Large Language Models (LLMs) (including GPT-4, Claude 3, Gemini, and Qwen families) leading to "Implicit Harm." Unlike traditional jailbreaks that use overtly harmful queries, this vulnerability allows remote attackers to coerce the model into providing factually incorrect, plausible, and dangerous responses to benign-looking inputs. By employing "JailFlip" techniques—specifically constructed affirmative-type or denial-type queries…

Beyond Jailbreaks: Revealing Stealthier and Broader LLM Security Risks Stemming from Alignment Failures

Evaluated models: GPT-4.1, GPT-4.1 Mini, GPT-4o+3 more

GPT-4.1
GPT-4.1 Mini
GPT-4o
GPT-4o Mini
Qwen Plus
Qwen Turbo

Nonsensical CoT Reasoning

Published May 23, 2025Analyzed Dec 30, 2025Source: arXiv

Large Language Models (LLMs) utilizing Chain-of-Thought (CoT) prompting are vulnerable to input perturbations that decouple intermediate reasoning from the final answer. An attacker can generate adversarial examples using gradient-based optimization (targeting specific loss functions that maximize reasoning divergence while minimizing answer loss) to induce "Right Answer, Wrong Reasoning" behaviors. This vulnerability manifests through two primary attack vectors: 1. Token-level perturbations…

Robust Answers, Fragile Logic: Probing the Decoupling Hypothesis in LLM Reasoning

Evaluated models: Llama 3 8B, Mistral 7B, Zephyr 7B Beta+4 more

Llama 3 8B
Mistral 7B
Zephyr 7B Beta
Qwen 2.5 7B
DeepSeek R1 Distill Qwen 7B
GPT-4o
GPT-3.5 Turbo

Confident Persuasion Override

Published Apr 1, 2025Analyzed Jan 14, 2026Source: arXiv

A vulnerability exists in Large Language Model (LLM) decision-making capabilities described as "Rhetorical Persuasion Override." When an LLM is deployed as a judge or evaluator in a single-turn, multi-agent debate framework, it systematically fails to distinguish factual truth from confidently presented misinformation. An adversarial agent can coerce the evaluator into endorsing a known falsehood from the TruthfulQA dataset by employing specific rhetorical strategies—namely, high confidence…

When persuasion overrides truth in multi-agent llm debates: Introducing a confidence-weighted persuasion override rate (cw-por)

Evaluated models: Llama 3.2 3B, Mistral 7B, Qwen 2.5 14B+1 more

Llama 3.2 3B
Mistral 7B
Qwen 2.5 14B
Phi-4 14B

Related research categories