Bimodal Black-Box Jailbreak
Published Analyzed Source: arXiv
A bimodal adversarial attack, PBI-Attack, can manipulate Large Vision-Language Models (LVLMs) into generating toxic or harmful content by iteratively optimizing both textual and visual inputs in a black-box setting. The attack leverages a surrogate LVLM to inject malicious features from a harmful corpus into a benign image, then iteratively refines both image and text perturbations to maximize the toxicity of the model’s output as measured by a toxicity detection model (Perspective API or…
BAMBA: A Bimodal Adversarial Multi-Round Black-Box Jailbreak Attacker for LVLMs
Evaluated models: GPT-4, InstructBLIP, MiniGPT-4+1 more
- GPT-4
- InstructBLIP
- MiniGPT-4
- Qwen VL