Sound Induces Trimodal Collapse
Published Analyzed Source: arXiv
A vulnerability exists in trimodal audio-video-language models where an attacker can systematically degrade multimodal reasoning through untargeted, audio-only adversarial perturbations. By optimizing a shared perturbation $\delta$ applied to the audio channel, an attacker can manipulate internal representations—specifically targeting audio encoder embeddings and cross-modal attention mechanisms—without modifying visual or textual inputs. The attack exploits the model's reliance on the audio…
SoundBreak: A Systematic Study of Audio-Only Adversarial Attacks on Trimodal Models
Evaluated models: VideoLLAMA2, Qwen 2 7B Instruct, Whisper Large-v2+1 more
- VideoLLAMA2
- Qwen 2 7B Instruct
- Whisper Large-v2
- PANNs