Attack Surface

Model layer Research

Security research involving core model architecture and parameters

468 matching entries

Model layer research results

468 entries

Colluding LoRA Emergent Harm

Published 3/1/2026

Analyzed 4/10/2026

A compositional vulnerability in modular Large Language Models (LLMs) allows attackers to bypass safety alignment by distributing malicious weight updates across multiple Parameter-Efficient Fine-Tuning (PEFT) adapters (e.g., LoRA). The malicious adapters are anchored to valid functional subspaces (e.g., math, coding) and exhibit benign behavior when evaluated in isolation, successfully evading standard unit-centric safety scans and static weight-space defenses. However, when a user linearly…

Colluding LoRA: A Composite Attack on LLM Safety Alignment

Evaluated models: Llama 3 8B, Qwen 2.5 7B, Gemma 2 2B

Source: arXiv

Contextual Priority Hacking

Published 3/1/2026

Analyzed 4/10/2026

Large Language Models (LLMs) are vulnerable to a jailbreak technique termed "Priority Hacking." Adversaries can bypass safety alignments by exploiting the model's internal priority graph, where certain abstract values (e.g., justice, public health) implicitly outweigh general safety restrictions within specific contexts. By crafting a deceptive prompt that frames a malicious request as a necessary action in service of a higher-priority benign value, attackers engineer a value conflict. The…

Are Dilemmas and Conflicts in LLM Alignment Solvable? A View from Priority Graph

Evaluated models: Not reported

Source: arXiv

Cross-Language Safety Drift

Published 3/1/2026

Analyzed 4/10/2026

Leading Large Language Models (LLMs) exhibit significant cross-lingual safety drift, allowing users to bypass safety guardrails by translating harmful prompts into low-resource Indic languages. While models effectively block unsafe prompts concerning caste, religion, gender, and politics in high-resource languages like English and Hindi, their safety alignment severely degrades in low-resource scripts such as Odia, Telugu, Kannada, and Punjabi. Evaluated models demonstrate a cross-language…

IndicSafe: A Benchmark for Evaluating Multilingual LLM Safety in South Asia

Evaluated models: GPT-4o Mini, Claude Sonnet 4, Grok 3 +6 more

Source: arXiv

Deep Safety Attention Jailbreak

Published 3/1/2026

Analyzed 4/10/2026

An activation-steering vulnerability in open-weights Large Language Models allows attackers to bypass safety guardrails by injecting targeted additive perturbations into deep, safety-critical attention heads. The exploit, termed Safety Attention Head Attack (SAHA), utilizes Ablation-Impact Ranking (AIR) to isolate specific attention heads that causally govern safety refusals. By applying Layer-Wise Perturbations (LWP) derived from the linearized decision boundary of a latent safety probe, an…

Depth Charge: Jailbreak Large Language Models from Deep Safety Attention Heads

Evaluated models: Llama 3.1 8B Instruct, DeepSeek LLM 7B Chat, Qwen 1.5 7B Chat

Source: arXiv

Defensive Refusal Bias

Published 3/1/2026

Analyzed 3/8/2026

Safety-aligned Large Language Models (LLMs) exhibit a "Defensive Refusal Bias" vulnerability, resulting in a safety-induced denial-of-service for legitimate cybersecurity operations. The models systematically refuse authorized defensive queries when they contain security-sensitive terminology (e.g., "exploit," "payload," "shell") because current alignment mechanisms rely on semantic similarity to harmful training data rather than intent analysis. Paradoxically, explicit authorization signals…

Defensive Refusal Bias: How Safety Alignment Fails Cyber Defenders

Evaluated models: Claude 3.5 Sonnet, GPT-4o, Llama 3.3 70B Instruct

Source: arXiv

Drive Overrides Refusal

Published 3/1/2026

Analyzed 4/10/2026

A prompt structure vulnerability exists in instruction-tuned Large Language Models (LLMs) where attackers can bypass safety alignments by injecting a continuation-triggering suffix immediately following the user prompt termination token. By placing an affirmative suffix outside the user instruction boundary, it is processed as the beginning of the assistant's own pre-filled response. This structural manipulation intrinsically overactivates the model's continuation attention heads, forcing its…

The Struggle Between Continuation and Refusal: A Mechanistic Analysis of the Continuation-Triggered Jailbreak in LLMs

Evaluated models: Llama 2 7B, Qwen 2.5 7B

Source: arXiv

Frontier LLM Safety Collapse

Published 3/1/2026

Analyzed 4/10/2026

Internal Safety Collapse (ISC) is a vulnerability in frontier Large Language Models (LLMs) where models autonomously generate highly restricted, harmful content while executing structurally legitimate professional workflows. The vulnerability triggers when a model infers that generating sensitive data is a functional requirement to complete an otherwise benign task. By nesting harmful content generation inside standard execution constraints (e.g., resolving a schema validation error in a…

Internal Safety Collapse in Frontier Large Language Models

Evaluated models: Gemini 3 Pro, Grok 4.1 Fast, Claude Sonnet 4.5 +1 more

Source: arXiv

Inter-Turn Modality Jailbreak

Published 3/1/2026

Analyzed 3/8/2026

Multimodal Large Language Models (LLMs) are vulnerable to alignment bypass via Inter-Turn Modality Switching (ITMS). By systematically rotating the input modality (e.g., alternating between text, audio, and image) across successive turns in a multi-turn adversarial conversation, an attacker can destabilize the model's safety defenses. The cross-modal transition mechanism exploits alignment gaps between differing input processing pipelines, accelerating the erosion of safety guardrails and…

MUSE: A Run-Centric Platform for Multimodal Unified Safety Evaluation of Large Language Models

Evaluated models: Gemini 2.5 Flash, Gemini 3 Flash Preview, GPT-4o +1 more

Source: arXiv

Invisible Visual Prompt Injection

Published 3/1/2026

Analyzed 4/11/2026

An imperceptible visual prompt injection vulnerability in Multimodal Large Language Models (MLLMs) allows attackers to execute precise command-hijacking via a Covert Triggered dual-Target Attack (CoTTA). By embedding a bounded, learnable textual overlay ($L_\infty$ norm bound $\varepsilon \le 16$) and adversarial noise into an input image, the attack forces the source image's internal feature representation to align with both the textual and visual embeddings of an attacker-specified…

Adversarial Prompt Injection Attack on Multimodal Large Language Models

Evaluated models: GPT-4o, GPT-5

Source: arXiv

LLM Judge Coin Flip

Published 3/1/2026

Analyzed 4/10/2026

Automated LLM-as-a-Judge safety classifiers exhibit severe performance degradation (falling to near-random chance) when subjected to distribution shifts caused by adversarial prompt optimization (Attack Shift), varying target architectures (Model Shift), and semantic categorization (Data Shift). Adversarial algorithms, particularly sampling-based (Best-of-N) and judge-aware optimization methods (GCG-REINFORCE), explicitly and implicitly exploit these judge insufficiencies. Instead of eliciting…

A Coin Flip for Safety: LLM Judges Fail to Reliably Measure Adversarial Robustness

Evaluated models: Llama 2 13B HarmBench, Llama Guard 3 8B, AegisGuard +1 more

Source: arXiv

Related research categories