Simulated Multi-Turn Priming Bypasses LLM Safety Alignment

Description

The paper reports a reproducible black-box jailbreak evaluation in which an evolutionary search generates simulated, role-labeled multi-turn dialogue histories and submits them as a single request. Five semantic mutators and graded judge feedback optimize conversational priming that can induce harmful responses despite direct-request refusals. Results are paper-reported and not independently verified.

Examples

See the primary source for the paper's evaluation and defensive reproduction context: https://arxiv.org/abs/2605.02647 (opens in a new tab)

Impact

The paper reports safety-alignment bypass across harmful-behavior categories including chemical/biological misuse, cybercrime, illegal activity, misinformation, and harassment. On 50 HarmBench behaviors, reported ASR@4 reached 90–100% for four open models; 40 attacks evolved against gpt-oss:120B reportedly transferred at 70–90% ASR@4 to three frontier models, while transfer to two Claude variants was 15–17.5%.

Affected Systems

• LLM inference endpoints accepting role-labeled multi-turn dialogue histories
• Public commercial LLM APIs
• Self-hosted LLM serving stacks

Mitigation Steps

• Treat user-supplied conversation histories, role delimiters, and purported prior assistant messages as untrusted; preserve turn provenance and reject fabricated privileged turns.
• Evaluate safety over the entire dialogue trajectory and final response, not only the final user request or surface keywords.
• Apply output moderation and refusal-consistency checks after contextual accumulation, including technical, troubleshooting, roleplay, and hypothetical framing.
• Rate-limit repeated semantically related attempts and monitor evolutionary probing patterns; the paper reports lower success at smaller query budgets for stronger targets.
• Run defensive regression tests using non-actionable substitute goals and the paper’s redacted artifacts; compare direct, flattened, and structured multi-turn formats without replaying harmful templates.
• Use multiple judges, human adjudication for sampled results, and evaluator-backdoor auditing because the paper identifies systematic judge bias as a limitation.

Evidence

• https://arxiv.org/abs/2605.02647 (opens in a new tab) (Abstract): the paper describes black-box evolutionary search over simulated multi-turn primed dialogue using five semantic mutators and a graded 0–5 two-level judge.
• arXiv:2605.02647 §3: the threat model requires only black-box target access; each query may contain multiple role-labeled turns submitted in a single API call, without logits, weights, training details, or defense knowledge.
• arXiv:2605.02647 §5.2 and Appendix G: the reported setup uses 50 HarmBench behaviors, a 100-attempt budget per goal, deterministic target decoding, qwen3-8B as mutator and barrier judge, and gpt-oss:120B as main judge.
• arXiv:2605.02647 §5.3: the paper reports 100% ASR@4 on gpt-oss:20B, llama3.1:70B, and qwen3-8B, and 90% on gpt-oss:120B; it also reports that gpt-oss:120B had 0% ASR@4 for direct requests but 72% ASR@5 under contextual prompting.
• arXiv:2605.02647 §5.5, Table 3: with semantic content held fixed, the paper reports mean ASR@4/ASR@5 of 37.5%/9.0% for direct requests, 73.5%/29.0% for flattened single sequences, and 97.5%/90.0% for structured context priming.
• arXiv:2605.02647 §5.8.2, Table 6: the paper reports verbatim transfer of 40 score-5 templates from gpt-oss:120B with ASR@4 of 90.0% on gpt-4o-mini, 70.0% on gemini-3-flash, 70.0% on gpt-5, 17.5% on claude-opus-4-7, and 15.0% on claude-sonnet-4-6.
• arXiv:2605.02647 Appendix A: the paper states that directly linked artifacts include source code, mutators, judges, configurations, prompts, benchmark data, and reproduction scripts, while withholding the complete successful-template database and providing only redacted synthetic examples.
• arXiv:2605.02647 §5.2: the authors explicitly identify systematic LLM-judge bias as a limitation and leave multi-judge agreement, human adjudication, and evaluator-backdoor auditing for future work.