Skip to main content
LLM Security Database
Skip to research details
Back to research findings
LMVD-ID: 7096f35e
Paper published August 1, 2024
Entry analyzed December 28, 2024
Paper-reported evidence
Confidence: Source-linked

The LMVD-ID is an internal research identifier, not an official CVE identifier.

Multi-Agent T2I Jailbreak

A vulnerability allows bypassing safety filters in text-to-image (T2I) models using a multi-agent framework ("Atlas") powered by Large Language Models (LLMs). Atlas iteratively generates and refines prompts, leveraging…

Application layer
Jailbreak
Multimodal
AI agents
Black-box
Safety
BibTeX citation

Paper-evaluated models(7)

DALL-E 3, LLaVA 1.5 13B, Sharegpt4v-13B +4 more
  • DALL-E 3
  • LLaVA 1.5 13B
  • Sharegpt4v-13B
  • Stable Diffusion 3 Medium
  • Stable Diffusion v1.4
  • Stable Diffusion Xl Refiner
  • Vicuna-1.5-13B

Description

A vulnerability allows bypassing safety filters in text-to-image (T2I) models using a multi-agent framework ("Atlas") powered by Large Language Models (LLMs). Atlas iteratively generates and refines prompts, leveraging a Vision-Language Model (VLM) to assess filter activation and an LLM to select effective prompts that maintain semantic similarity to the original, malicious prompt while evading the filter. This enables the generation of images containing unsafe content.

Examples

Specific examples of successful jailbreak prompts and resulting images are available in the associated research paper. See arXiv:2408.00523 (opens in a new tab).

Impact

Successful exploitation allows bypassing safety mechanisms in T2I models, leading to the generation of unsafe content such as NSFW images, violent content, or images violating terms of service. This impacts the safety and ethical implications of using T2I models.

Affected Systems

Multiple state-of-the-art text-to-image models (Stable Diffusion v1.4, Stable Diffusion XL, Stable Diffusion 3, DALL-E 3) with various safety filters are affected. The vulnerability is demonstrated across various types of safety filters (text-based, image-based, text-image-based) showing wide applicability.

Mitigation Steps

  • Enhance safety filter training with adversarial examples generated by similar LLM-based agents.
  • Implement robust prompt sanitization techniques designed to detect and mitigate iterative prompt manipulation strategies.
  • Employ rate limiting or account suspension mechanisms for users exhibiting suspicious prompt patterns indicative of jailbreaking attempts.
  • Explore and develop certification mechanisms for T2I Models to verify robustness against prompt-based attacks.

Research context and confidence

Evidence and verification
Paper-reported; independent reproduction is not documented.
Primary research source linked.
Severity
Not rated by this catalog.
Source and publication type
arXiv · Research preprint.
Peer-review status is not provided by this source.
Author and publication status
Author metadata is not stored; see the primary paper.
Threat model and attacker access
Black-box model, service, or application access.
Related deployment categories
Agent workflows
Taxonomy labels only; paper-specific deployment prerequisites are not inferred.
Affected systems
Multiple state-of-the-art text-to-image models (Stable Diffusion v1.4, Stable Diffusion XL, Stable Diffusion 3, DALL-E 3) with various safety filters are affected. The vulnerability is demonstrated across various types…

Research Paper

Jailbreaking text-to-image models with llm-based agents

Primary source: arXiv. Findings are reported by the cited research and have not been independently verified.

View Paper

Evidence

This entry is based on a primary research source. Its findings are paper-reported; independent reproduction and verification are not claimed.

https://arxiv.org/abs/2408.00523

Related research