Impact

Reliability Research

Issues affecting model consistency and dependability

4 matching entries out of 203 in this category

Reliability research results

4 entries

Voice Agent Behavioral Bypass

Published Feb 7, 2026Analyzed Feb 21, 2026Source: arXiv

Audio Large Language Models (ALLMs) integrated into voice agent systems for high-stakes domains (banking, IT support, logistics) are vulnerable to multimodal adversarial attacks via spoken interaction. Adversaries can exploit the model's inherent compliance and contextual awareness through multi-turn dialogue to bypass authentication safeguards, escalate privileges (e.g., unauthorized credit limit increases), exfiltrate sensitive Personally Identifiable Information (PII), and poison…

Aegis: Towards Governance, Integrity, and Security of AI Voice Agents

Evaluated models: GPT-4o, GPT-4o Mini, Gemini 1.5 Pro+4 more

GPT-4o
GPT-4o Mini
Gemini 1.5 Pro
Gemini 2.5 Flash
Gemini 2.5 Pro
Qwen2-Audio 7B
Qwen 2.5 Omni 7B

Hybrid Agent Prompt Injection

Published May 28, 2025Analyzed Dec 9, 2025Source: arXiv

Computer-Use Agents (CUAs) powered by Large Language Models (LLMs) operating in hybrid Web-OS environments are vulnerable to indirect prompt injection. Attackers can embed malicious natural language or code instructions within legitimate web content (e.g., social media forums, chat applications, shared cloud documents) that the agent processes during benign task execution. Due to the agent's inability to distinguish between trusted user instructions and untrusted environmental data, the CUA…

RedTeamCUA: Realistic Adversarial Testing of Computer-Use Agents in Hybrid Web-OS Environments

Evaluated models: Claude 3.5 Sonnet, Claude 3.7 Sonnet, GPT-4o

On-Device LLM Hijacking

Published May 19, 2025Analyzed Dec 9, 2025Source: arXiv

Mobile LLM agents utilizing vision-based screen perception (OCR or Multimodal Large Language Models) are vulnerable to Visual Prompt Injection via malicious GUI overlays. An attacker holding the SYSTEM_ALERT_WINDOW permission can deploy non-focusable floating windows (using FLAG_NOT_FOCUSABLE) containing adversarial text or fabricated UI elements over legitimate applications. Because the agent captures the entire screen buffer to interpret the device state, it ingests the adversarial overlay…

From Assistants to Adversaries: Exploring the Security Risks of Mobile LLM Agents

Evaluated models: GPT-4o

LLM Judge Adversarial Vulnerability

Published Mar 6, 2025Analyzed Mar 19, 2025Source: arXiv

Large Language Model (LLM) safety judges exhibit vulnerability to adversarial attacks and stylistic prompt modifications, leading to increased false negative rates (FNR) and decreased accuracy in classifying harmful model outputs. Minor stylistic changes to model outputs, such as altering the formatting or tone, can significantly impact a judge's classification, while direct adversarial modifications to the generated text can fool judges into misclassifying even 100% of harmful generations as…

Know Thy Judge: On the Robustness Meta-Evaluation of LLM Safety Judges

Evaluated models: Atla Selene Mini 8B, Llama 2 13B, Llama 3.1 8B+4 more

Atla Selene Mini 8B
Llama 2 13B
Llama 3.1 8B
Llama Guard 3 8B
Mistral 7B
ShieldGemma 9B
WildGuard

Related research categories