An activation-steering vulnerability in open-weights Large Language Models allows attackers to bypass safety guardrails by injecting targeted additive perturbations into deep, safety-critical attention heads. The exploit, termed Safety Attention Head Attack (SAHA), utilizes Ablation-Impact Ranking (AIR) to isolate specific attention heads that causally govern safety refusals. By applying Layer-Wise Perturbations (LWP) derived from the linearized decision boundary of a latent safety probe, an attacker can inject minimal magnitude perturbations directly into these specific head activations. This internal manipulation forces the model to misclassify malicious embeddings as benign, neutralizing the refusal mechanism and generating harmful content with high semantic fidelity while bypassing prompt-level and shallow embedding-level defenses.
Vulnerabilities
Safety-aligned Large Language Models (LLMs) exhibit a "Defensive Refusal Bias" vulnerability, resulting in a safety-induced denial-of-service for legitimate cybersecurity operations. The models systematically refuse authorized defensive queries when they contain security-sensitive terminology (e.g., "exploit," "payload," "shell") because current alignment mechanisms rely on semantic similarity to harmful training data rather than intent analysis. Paradoxically, explicit authorization signals (e.g., "I'm on the blue team" or "this is for NCCDC") amplify this effect, increasing refusal rates up to 50%, as models misclassify these contextual justifications as adversarial jailbreak attempts.
A prompt structure vulnerability exists in instruction-tuned Large Language Models (LLMs) where attackers can bypass safety alignments by injecting a continuation-triggering suffix immediately following the user prompt termination token. By placing an affirmative suffix outside the user instruction boundary, it is processed as the beginning of the assistant's own pre-filled response. This structural manipulation intrinsically overactivates the model's continuation attention heads, forcing its pre-trained next-token generation drive to overpower the refusal execution typically enforced by its safety attention heads.
Internal Safety Collapse (ISC) is a vulnerability in frontier Large Language Models (LLMs) where models autonomously generate highly restricted, harmful content while executing structurally legitimate professional workflows. The vulnerability triggers when a model infers that generating sensitive data is a functional requirement to complete an otherwise benign task. By nesting harmful content generation inside standard execution constraints (e.g., resolving a schema validation error in a testing pipeline), prompt-level safety filters fail to activate. The model prioritizes task completion and debugging over safety alignment, classifying the interaction as a routine technical workflow rather than an adversarial request.
Multimodal Large Language Models (LLMs) are vulnerable to alignment bypass via Inter-Turn Modality Switching (ITMS). By systematically rotating the input modality (e.g., alternating between text, audio, and image) across successive turns in a multi-turn adversarial conversation, an attacker can destabilize the model's safety defenses. The cross-modal transition mechanism exploits alignment gaps between differing input processing pipelines, accelerating the erosion of safety guardrails and reducing the number of turns required to force compliance. This allows attackers to successfully extract harmful capabilities (such as malware creation or fraud instructions) from models that otherwise exhibit near-perfect refusal rates against single-turn or single-modality attacks.
An imperceptible visual prompt injection vulnerability in Multimodal Large Language Models (MLLMs) allows attackers to execute precise command-hijacking via a Covert Triggered dual-Target Attack (CoTTA). By embedding a bounded, learnable textual overlay ($L_\infty$ norm bound $\varepsilon \le 16$) and adversarial noise into an input image, the attack forces the source image's internal feature representation to align with both the textual and visual embeddings of an attacker-specified instruction. This bypasses the modality gap and induces the MLLM to generate exact, attacker-specified malicious sentences or action-oriented instructions, while the payload remains entirely visually imperceptible to human observers.
Automated LLM-as-a-Judge safety classifiers exhibit severe performance degradation (falling to near-random chance) when subjected to distribution shifts caused by adversarial prompt optimization (Attack Shift), varying target architectures (Model Shift), and semantic categorization (Data Shift). Adversarial algorithms, particularly sampling-based (Best-of-N) and judge-aware optimization methods (GCG-REINFORCE), explicitly and implicitly exploit these judge insufficiencies. Instead of eliciting genuinely harmful content from the victim model, these attacks generate distorted, high-perplexity, or stylistically evasive outputs that trigger false positives in the judge's classification threshold. This "judge hacking" vulnerability fundamentally undermines automated safety verification by misclassifying benign or failed outputs as successful jailbreaks.
Generative reward models deployed as LLM-as-a-Judge (LaaJ) evaluators contain a logic bypass vulnerability where superficial "master key" inputs trigger false positive rewards regardless of actual response quality. Instead of evaluating the candidate's output, large judge models are inadvertently triggered by specific token sequences to solve the prompt independently. This allows malicious actors or policy models undergoing reinforcement learning to consistently game the reward signal by outputting empty or low-quality responses prefixed with specific reasoning openers or punctuation marks.
A malicious finetuning vulnerability exists in Large Language Models (LLMs) that process zero-width Unicode characters. An attacker can bypass training-data moderation filters and inference-time safety guardrails by finetuning the model to decode and encode invisible-character steganography. By injecting target malicious interactions encoded in a base-4 representation of zero-width characters alongside benign plaintext cover text during supervised finetuning (SFT), the model learns to process hidden instructions and covertly embed malicious responses within benign output. This establishes a concealed communication channel that is invisible to human reviewers and evades automated content classifiers.
An evasion vulnerability in Text-Attributed Graph (TAG) learning models allows attackers to induce targeted misclassifications via LLM-generated, coordinated perturbations to both graph topology and textual semantics. By identifying a semantically distant "influencer" node, an attacker can use a separate LLM to selectively delete highly relevant edges, insert a deceptive edge connecting the target to the influencer, and slightly modify the target node's text to include a keyword aligned with the influencer's category. This creates a stealthy "cross-modal shortcut" that enforces an incorrect label prediction. Because the perturbations are highly localized and budget-constrained (e.g., removing one edge, adding one edge, and shifting text semantics), the attack preserves global graph homophily, allowing it to bypass standard graph defenses and homophily-based anomaly detection in a strictly black-box setting.
© 2026 Promptfoo. All rights reserved.