Attack Surface

Model layer Research

Security research involving core model architecture and parameters

468 matching entries

Model layer research results

468 entries

Dynamic Backtracking Failure

Published 2/1/2026

Analyzed 2/22/2026

Large Language Models (LLMs) aligned via standard Reinforcement Learning from Human Feedback (RLHF) or Supervised Fine-Tuning (SFT) are vulnerable to "Shallow Safety" bypass attacks, specifically Middle Filling (MF) and Greedy Coordinate Gradient (GCG) attacks. These models frequently rely on refusal mechanisms triggered solely by the initial tokens of a prompt. By embedding malicious instructions after a benign context (prefilling) or utilizing suffix optimization, attackers can induce the…

Reinforcement Learning with Backtracking Feedback

Evaluated models: Llama 3.2 1B, Llama 3.2 3B, Llama 3 8B Instruct +2 more

Source: arXiv

Early Reasoning Safety Degradation

Published 2/1/2026

Analyzed 3/8/2026

Reinforcement learning (RL) based post-training for explicit chain-of-thought reasoning (e.g., GRPO) in Multimodal Large Reasoning Models (MLRMs) inadvertently degrades safety alignment, rendering the models highly vulnerable to multimodal jailbreak attacks. The vulnerability is caused by "conditional coverage collapse" during the initial phases of chain-of-thought generation. Under adversarial conditioning (text or image), the reasoning policy assigns vanishing probability mass to safe…

Safety Recovery in Reasoning Models Is Only a Few Early Steering Steps Away

Evaluated models: R1-Onevision 7B, OpenVLThinker 7B, VLAA-Thinker 7B +3 more

Source: arXiv

Emergent Distractor Jailbreak

Published 2/1/2026

Analyzed 3/9/2026

A vulnerability in the Direct Preference Optimization (DPO) post-training phase of OLMo 2 models leads to "distractor-triggered compliance." The model correctly refuses harmful requests when prompted in isolation, but complies with identical harmful requests if a benign formatting instruction (a "distractor") is appended to the prompt. This behavior organically emerges from contaminated preference training data where mislabeled examples incorrectly preferred compliance over refusal when a…

In-the-Wild Model Organisms: Mitigating Undesirable Emergent Behaviors in Production LLM Post-Training via Data Attribution

Evaluated models: Not reported

Source: arXiv

Enhanced LVLM Encoder Transfer

Published 2/1/2026

Analyzed 4/11/2026

Large Vision-Language Models (LVLMs) are vulnerable to zero-query, black-box adversarial image perturbations via Semantic-Guided Multimodal Attacks (SGMA). Unlike traditional attacks that scatter noise or target background pixels, SGMA leverages surrogate models (e.g., CLIP) to anchor imperceptible adversarial perturbations directly onto semantically critical foreground regions. The attack exploits two specific architectural traits of LVLMs: inconsistent visual grounding across models and…

Grounding-Driven Attack: Improving Encoder-based Adversarial Transferability against Large Vision-Language Models

Evaluated models: BLIP-2 OPT 2.7B, LLaVA 1.5 7B, Qwen 2.5 VL 7B Instruct +6 more

Source: arXiv

Evolutionary Hidden Knowledge Recovery

Published 2/1/2026

Analyzed 2/21/2026

Large Language Models (LLMs) subjected to machine unlearning techniques (specifically AltPO, GradDiff, IDKDPO, IDKNLL, UNDIAL, NPO, and SimNPO) contain a vulnerability regarding the persistence of latent knowledge. Despite achieving high "forgetting" scores on standard, benign benchmarks, these models remain susceptible to black-box evolutionary adversarial attacks. An attacker can utilize an automated framework (REBEL) comprising a "Hacker" model and a "Judge" model to iteratively mutate…

REBEL: Hidden Knowledge Recovery via Evolutionary-Based Evaluation Loop

Evaluated models: Not reported

Source: arXiv

Gradient Shift Jailbreak

Published 2/1/2026

Analyzed 2/20/2026

Large Language Models (LLMs) and Large Reasoning Models (LRMs) are vulnerable to an inference-time jailbreak attack known as "Hybrid Gradient Shift." This vulnerability exploits the statistical discrepancy between an aligned model's output distribution and a pre-alignment reference distribution. By treating safety alignment as a form of miscalibration, an attacker can aggregate the logits of a target model (aligned), a helper model (unaligned/weak), and a predictor model to reconstruct the…

Jailbreaking LLMs via Calibration

Evaluated models: GPT-oss 120B, Llama 3.3 70B Instruct

Source: arXiv

Heavy Reasoning Filter Bypass

Published 2/1/2026

Analyzed 3/8/2026

Multiple open-source Large Language Models (LLMs) in the 1B to 7B parameter range are vulnerable to safety bypasses via long-form, multi-step reasoning prompt injection and jailbreak attacks. Attackers can evade alignment by embedding malicious instructions within extended contextual narratives, exploiting "attention dilution" and the models' tendency to prioritize contextual coherence over safety constraints (semantic camouflage). These reasoning-heavy attacks consistently bypass standard…

Analysis of LLMs Against Prompt Injection and Jailbreak Attacks

Evaluated models: Phi-3 Mini 3.8B, Mistral 7B, DeepSeek R1 Distill Qwen 7B +7 more

Source: arXiv

High-Dim Bandit Reward Hijack

Published 2/1/2026

Analyzed 3/9/2026

A vulnerability in offline multi-armed bandit (MAB) evaluation frameworks allows attackers to completely hijack arm selection by applying imperceptibly small adversarial perturbations to the weights of the underlying reward model before training begins. Because the required perturbation norm decreases as the input dimensionality increases ($\mathcal{O}(d^{-1/2})$), high-dimensional reward models—such as wide neural networks or linear models used for generative model assessment—are highly…

Efficient Adversarial Attacks on High-dimensional Offline Bandits

Evaluated models: Stable Diffusion

Source: arXiv

Implicit Memory Covert Channel

Published 2/1/2026

Analyzed 3/9/2026

A vulnerability exists in Large Language Models (LLMs) deployed in environments with output reingestion (e.g., RAG, coding assistants, agentic workflows) that allows attackers to execute "temporal backdoors" (time bombs) via an implicit memory channel. Attackers can implant this behavior via system prompts or fine-tuning (data poisoning) to make the model encode hidden state information within its generated text using non-printing Unicode characters or semantic steganography. When these…

Position: Stateless Yet Not Forgetful: Implicit Memory as a Hidden Channel in LLMs

Evaluated models: o3-mini, o4-mini, GPT-oss 120B +7 more

Source: arXiv

Indic Attack Amplification

Published 2/1/2026

Analyzed 2/21/2026

Large Language Models (LLMs), specifically those aligned primarily using English-centric data (such as LLaMA-3-8B-Instruct, GPT-OSS 20B, and Qwen3-32B), contain a cross-lingual safety generalization vulnerability. Safety guardrails and refusal logic fail to transfer effectively to linguistically distant languages, particularly Indic languages (Hindi, Assamese, Marathi, Kannada, and Gujarati). This vulnerability allows attackers to bypass safety alignment by translating structured adversarial…

Lost in Translation? A Comparative Study on the Cross-Lingual Transfer of Composite Harms

Evaluated models: Llama 3 8B Instruct, GPT-oss 20B, Qwen 3 32B

Source: arXiv

Related research categories