Skip to main content
LLM Security Database
Skip to research details
Back to research findings
LMVD-ID: b02f4c07
Paper published November 1, 2024
Entry analyzed December 29, 2024
Paper-reported evidence
Confidence: Source-linked

The LMVD-ID is an internal research identifier, not an official CVE identifier.

VLM RedTeaming Jailbreak

Large Vision-Language Models (VLMs) are vulnerable to a novel black-box jailbreak attack, IDEATOR, which leverages a separate VLM to generate malicious image-text pairs. The attacker VLM iteratively refines its prompts…

Jailbreak
Multimodal
Black-box
Vision
AI agents
Safety
BibTeX citation

Paper-evaluated models(13)

MiniGPT-4 Vicuna 13B, InstructBLIP, Chameleon +10 more
  • MiniGPT-4 Vicuna 13B
  • InstructBLIP
  • Chameleon
  • LLaVA-OneVision
  • MiniGPT-v2
  • Llama 3.2 11B Vision
  • Llama 3.2 90B Vision
  • GPT-4o Mini
  • GPT-4o
  • Gemini 1.5 Pro
  • Gemini 2.0 Flash
  • Gemini 2.0 Flash Thinking
  • Claude 3.5 Sonnet

Description

Large Vision-Language Models (VLMs) are vulnerable to a novel black-box jailbreak attack, IDEATOR, which leverages a separate VLM to generate malicious image-text pairs. The attacker VLM iteratively refines its prompts based on the target VLM's responses, bypassing safety mechanisms by generating contextually relevant and visually subtle malicious prompts.

Examples

See the IDEATOR paper (the provided text). Specific examples of successful attacks against MiniGPT-4, LLaVA, InstructBLIP, and Meta's Chameleon are detailed within along with the generated image-text pairs.

Impact

Successful exploitation allows attackers to bypass built-in safety restrictions of VLMs, eliciting harmful outputs (e.g., instructions for illegal activities, hate speech, disinformation). The high success rate (94% against MiniGPT-4) and transferability across different VLMs highlight the severity of this vulnerability.

Affected Systems

Large Vision-Language Models (VLMs), including but not limited to MiniGPT-4, LLaVA, InstructBLIP, and Meta's Chameleon. Other VLMs employing similar architectures and safety mechanisms are likely affected.

Mitigation Steps

  • Implement more robust safety mechanisms that are resistant to iterative adversarial attacks.
  • Develop detection methods for identifying and blocking malicious image-text pairs generated by techniques such as IDEATOR.
  • Further research into improving the robustness of VLMs against adversarial attacks is needed. Regularly evaluate and update safety mechanisms.

Research context and confidence

Evidence and verification
Paper-reported; independent reproduction is not documented.
Primary research source linked.
Severity
Not rated by this catalog.
Source and publication type
arXiv · Research preprint.
Peer-review status is not provided by this source.
Author and publication status
Author metadata is not stored; see the primary paper.
Threat model and attacker access
Black-box model, service, or application access.
Related deployment categories
Vision-language models; Agent workflows
Taxonomy labels only; paper-specific deployment prerequisites are not inferred.
Affected systems
Large Vision-Language Models (VLMs), including but not limited to MiniGPT-4, LLaVA, InstructBLIP, and Meta's Chameleon. Other VLMs employing similar architectures and safety mechanisms are likely affected.

Research Paper

IDEATOR: Jailbreaking and Benchmarking Large Vision-Language Models Using Themselves

Primary source: arXiv. Findings are reported by the cited research and have not been independently verified.

View Paper

Evidence

This entry is based on a primary research source. Its findings are paper-reported; independent reproduction and verification are not claimed.

https://arxiv.org/abs/2411.00827

Related research