Jailbreaks Causally Suppress Refusal Representations

Description

The paper reports a reproducible white-box evaluation showing that successful jailbreak prompts can alter a safety-aligned model’s intermediate representations so harmful requests no longer trigger refusal. Its LOCA method identifies small, token-specific residual-stream changes that restore refusal on individual successful jailbreaks, providing causal evidence that jailbreak success can depend on suppressing harmfulness/refusal concepts or strengthening seemingly harmless continuation concepts. These are paper-reported findings, not independently verified facts.

Examples

See the primary source for the paper's evaluation and defensive reproduction context: https://arxiv.org/abs/2605.00123 (opens in a new tab)

Impact

Successful jailbreaks can cause safety-aligned chat models to answer harmful requests that they refuse in canonical form, undermining safety and response reliability. The paper’s case study attributes one success to suppressed harmful-request signaling and strengthened harmless/code-generation-related signaling. The authors also warn that their first-token refusal proxy can miss harmful content emitted later in a response.

Affected Systems

• Safety-aligned, instruction-tuned decoder-only chat LLMs with accessible residual-stream activations
• Gemma-2-2B-IT
• Llama-3.1-8B-Instruct

Mitigation Steps

• Use diverse, category-specific jailbreak evaluations rather than assuming one global refusal mechanism covers all attack strategies and harmful-request categories.
• For open-weight models, apply LOCA-style activation-patching analysis to successful jailbreak cases to identify the minimal token/concept changes that restore refusal and use those findings to guide targeted safety improvements.
• Validate refusal over the full generated response, not only the first output token, because the paper documents cases where a response begins like a refusal but later contains harmful content.
• Red-team chat-template, punctuation, and post-instruction token behavior, since the paper reports these positions can be causally important to refusal in middle layers.
• Treat sparse-autoencoder concept interpretations as diagnostic hypotheses requiring manual review; the authors report that early-layer interpretations are time-intensive, error-prone, and sometimes unclear.

Evidence

• https://arxiv.org/abs/2605.00123 (opens in a new tab) — abstract and submission history: submitted 2026-04-30; the authors state that LOCA identifies minimal interpretable intermediate-representation changes that causally induce refusal on otherwise successful jailbreak requests.
• https://arxiv.org/html/2605.00123 (opens in a new tab) — Sections 3 and 3.5: LOCA token-matches original and jailbreak prompts, ranks token/concept changes with a first-order approximation, and iteratively activation-patches residual-stream representations, making the evaluation method reproducible for white-box models with suitable SAEs.
• https://arxiv.org/html/2605.00123 (opens in a new tab) — Section 4: the paper evaluates Gemma-2-2B-IT and Llama-3.1-8B-Instruct using the WhatFeatures dataset of 10,800 attacks from 35 jailbreak methods, greedy decoding, and HarmBench labels.
• https://arxiv.org/html/2605.00123 (opens in a new tab) — Section 4.1 and Figure 3: the authors report that LOCA induces refusal on Llama with an average of 6–8 early-layer patches and on Gemma with 12–16, while refusal rate reaches 100% at particular evaluated layers; these are measurements reported by the paper, not independently verified here.
• https://arxiv.org/html/2605.00123 (opens in a new tab) — Section 4.4: in an AutoDAN case study on Llama, the authors report that two layer-11 activation patches restored refusal and interpret the jailbreak as suppressing a harmful-request/refusal concept while strengthening a harmless code-generation-related concept.
• https://arxiv.org/html/2605.00123 (opens in a new tab) — Section 4.3 and Figure 5: the paper reports that early-layer refusal-relevant changes tend to involve user-instruction tokens, whereas middle-layer changes shift toward punctuation and post-instruction/chat-template tokens.
• https://arxiv.org/html/2605.00123 (opens in a new tab) — Section 5 and Appendix F: the authors state that interpretations can be error-prone and document failure cases where their first-output-token refusal proxy misses later harmful continuation, limiting the strength of the evaluation claim.
• https://arxiv.org/html/2605.00123 (opens in a new tab) — Appendix D: the paper reports experiments completed on a single NVIDIA A40 or A100 in under 10 GPU-hours and reports approximately 7 seconds to generate a LOCA local explanation under its setup.