Architecture Components

Audio Research

Security issues in speech and audio-enabled AI systems

17 matching entries out of 19 in this category

Audio research results

17 entries

Joint Audio-Text Jailbreak

Published Mar 19, 2026Analyzed Apr 10, 2026Source: arXiv

Spoken Language Models (SLMs) are vulnerable to Joint Audio-text Multimodal Attacks (JAMA), which bypass safety alignments by simultaneously perturbing both text and audio inputs. The vulnerability exploits the combined optimization of a discrete text suffix via Greedy Coordinate Gradient (GCG) and a continuous audio perturbation via Projected Gradient Descent (PGD). This joint gradient-based attack pushes the model's hidden layer representations into a distinct subspace far from the benign…

On Optimizing Multimodal Jailbreaks for Spoken Language Models

Evaluated models: Qwen2-Audio 7B Instruct, Qwen 2.5 Omni 7B, Audio Flamingo 3+1 more

Qwen2-Audio 7B Instruct
Qwen 2.5 Omni 7B
Audio Flamingo 3
Gemma 3n E2B

Inaudible Ultrasonic LLM Jailbreak

Published Mar 14, 2026Analyzed Apr 10, 2026Source: arXiv

Speech-driven Large Language Models (LLMs) and end-to-end Large Audio-Language Models (LALMs) are vulnerable to inaudible near-ultrasonic prompt injections, a framework dubbed Sirens' Whisper (SWhisper). By exploiting the non-linear response of commodity microphones, attackers can encode structured, phonetically optimized adversarial prompts into the 17–22 kHz near-ultrasonic band. Using regularized channel-inversion pre-compensation, the attacker shapes the waveform to account for microphone…

Sirens' Whisper: Inaudible Near-Ultrasonic Jailbreaks of Speech-Driven LLMs

Evaluated models: GLM-4 Voice, Qwen Omni Turbo, Llama 3.1 8B Instruct+5 more

GLM-4 Voice
Qwen Omni Turbo
Llama 3.1 8B Instruct
Gemma 3 4B
Qwen 2.5 7B Instruct
Mistral 7B Instruct v0.3
GLM-4 Air 250414
Grok 4

Voice Agent Behavioral Bypass

Published Feb 7, 2026Analyzed Feb 21, 2026Source: arXiv

Audio Large Language Models (ALLMs) integrated into voice agent systems for high-stakes domains (banking, IT support, logistics) are vulnerable to multimodal adversarial attacks via spoken interaction. Adversaries can exploit the model's inherent compliance and contextual awareness through multi-turn dialogue to bypass authentication safeguards, escalate privileges (e.g., unauthorized credit limit increases), exfiltrate sensitive Personally Identifiable Information (PII), and poison…

Aegis: Towards Governance, Integrity, and Security of AI Voice Agents

Evaluated models: GPT-4o, GPT-4o Mini, Gemini 1.5 Pro+4 more

GPT-4o
GPT-4o Mini
Gemini 1.5 Pro
Gemini 2.5 Flash
Gemini 2.5 Pro
Qwen2-Audio 7B
Qwen 2.5 Omni 7B

Audio Narrative Jailbreak

Published Jan 30, 2026Analyzed Feb 21, 2026Source: arXiv

End-to-end Large Audio-Language Models (LALMs) are vulnerable to paralinguistic jailbreak attacks where the acoustic delivery style of an input—specifically tone, prosody, and emotional framing—overrides safety alignment mechanisms. Unlike adversarial perturbations that inject noise, this vulnerability exploits the model's personification bias by utilizing standard Text-to-Speech (TTS) synthesis to render prohibited instructions in psychologically manipulative vocal styles (e.g…

Now You Hear Me: Audio Narrative Attacks Against Large Audio-Language Models

Evaluated models: GPT-4o Realtime, Gemini 2.0 Flash, Qwen 2.5 Omni 7B

Speech-Audio Composition Attack

Published Nov 13, 2025Analyzed Dec 30, 2025Source: arXiv

Multimodal Large Language Models (MLLMs) capable of processing speech and audio are vulnerable to Speech-Audio Compositional Attacks. This vulnerability exists because current safety mechanisms often rely on text-only transcription or fail to analyze the full acoustic context of an input. By manipulating the composition of audio signals, an attacker can bypass safety filters and elicit harmful responses. The attacks exploit three specific mechanisms: (1) Speech Overlap, where harmful…

Speech-Audio Compositional Attacks on Multimodal LLMs and Their Defense with SALMONN-Guard

Evaluated models: Qwen2-Audio 7B, Qwen 2.5 Omni 7B, Step-Audio 2 Mini Base+6 more

Qwen2-Audio 7B
Qwen 2.5 Omni 7B
Step-Audio 2 Mini Base
MiniCPM-o 2.6 8B
Qwen3-Omni 30B-A3B Instruct
Kimi-Audio 7B Instruct
Gemini 1.5 Pro
GPT-4o
Gemini 2.5 Pro

LAM Speech Style Jailbreak

Published Nov 12, 2025Analyzed Dec 8, 2025Source: arXiv

Large Audio-Language Models (LAMs) are vulnerable to style-aware audio jailbreak attacks that bypass safety alignment mechanisms. This vulnerability exists because current safety alignment strategies often overlook the expressive variations of human speech. Attackers can exploit this by manipulating three specific attributes of the audio input: linguistic (rewriting text with emotional semantics), paralinguistic (modulating emotional acoustic tone), and extralinguistic (altering speaker age…

StyleBreak: Revealing Alignment Vulnerabilities in Large Audio-Language Models via Style-Aware Audio Jailbreak

Evaluated models: GPT-4o, Llama 3.1 8B, Qwen 2 7B+1 more

GPT-4o
Llama 3.1 8B
Qwen 2 7B
Qwen 2.5 7B

Benign Audio Jailbreak

Published Aug 5, 2025Analyzed Dec 9, 2025Source: arXiv

Audio-Language Models (ALMs) including Qwen2.5-Omni (3B and 7B) and Phi-4-Multimodal are vulnerable to "WhisperInject," a two-stage adversarial audio attack that bypasses safety guardrails. The vulnerability allows an attacker to inject imperceptible perturbations into benign audio inputs (e.g., a query about the weather) that force the model to generate specific harmful content. The attack utilizes a novel optimization method, Reinforcement Learning with Projected Gradient Descent (RL-PGD)…

When Good Sounds Go Adversarial: Jailbreaking Audio-Language Models with Benign Inputs

Evaluated models: Qwen 2.5 Omni 3B, Qwen 2.5 Omni 7B, Phi-4 Multimodal+2 more

Qwen 2.5 Omni 3B
Qwen 2.5 Omni 7B
Phi-4 Multimodal
Gemma 3n E2B
Gemma 3n E4B

Airborne Audio LLM Hijack

Published Jul 7, 2025Analyzed Jan 14, 2026Source: arXiv

Audio-based Large Language Models (ALLMs), specifically Qwen2-Audio, are vulnerable to over-the-air adversarial audio attacks. An attacker with white-box access can generate robust adversarial audio perturbations using gradient-based optimization combined with audio augmentation techniques (specifically SpecAugment, translation, and additive noise). These perturbations, when played through a speaker in the physical environment, manipulate the ALLM processing the audio via a microphone. This…

Attacker's Noise Can Manipulate Your Audio-based LLM in the Real World

Evaluated models: Not reported

Audio Adversarial Jailbreak

Published May 24, 2025Analyzed May 31, 2025Source: arXiv

Research ID daca2ed7

A vulnerability in SpeechGPT allows bypassing safety filters through adversarial audio prompts crafted by a white-box token-level attack. The attacker leverages knowledge of SpeechGPT's internal speech tokenization process to generate adversarial token sequences, which are then synthesized into audio. These audio prompts elicit restricted or harmful outputs the model would normally suppress. The attack's effectiveness relies on the model's discrete audio token representation and does not…

Audio Jailbreak Attacks: Exposing Vulnerabilities in SpeechGPT in a White-Box Framework

Evaluated models: SpeechGPT

Semantic Audio Jailbreak

Published May 21, 2025Analyzed Dec 8, 2025Source: arXiv

Large Audio-Language Models (LAMs) are vulnerable to adversarial signal-level perturbations that allow for the bypass of safety guardrails (jailbreaking). While these models may possess robust text-based safety alignment, they fail to generalize this robustness to the audio modality. Attackers can utilize the Audio Perturbation Toolkit (APT) to apply transformations in the time domain (Energy Distribution Perturbation, Trimming, Fade In/Out), frequency domain (Pitch Shifting, Temporal…

Audio Jailbreak: An Open Comprehensive Benchmark for Jailbreaking Large Audio-Language Models

Evaluated models: SpeechGPT, SALMONN, DiVA+4 more

SpeechGPT
SALMONN
DiVA
Qwen 2 Audio
Llama Omni
Gemini 2.0 Flash
GPT-4o Audio

Related research categories