Attack Surface

Model layer Research

Security research involving core model architecture and parameters

468 matching entries

Model layer research results

468 entries

Indic Language Jailbreak

Published 2/1/2026

Analyzed 3/8/2026

Multiple large language models are vulnerable to cross-lingual and orthographic jailbreaks utilizing South Asian (Indic) languages. Attackers can bypass safety alignment and elicit harmful content by formulating requests in native Indic scripts (e.g., Bengali, Odia, Urdu) or by utilizing cross-lingual transfer attacks where English adversarial wrappers (format or instruction overrides) encapsulate Indic-language targets. Evaluations reveal a severe "contract gap": while imposing strict JSON…

IndicJR: A Judge-Free Benchmark of Jailbreak Robustness in South Asian Languages

Evaluated models: Command A, Command R, Gemma 2 9B +9 more

Source: arXiv

LLM Backdoor Trigger Reconstruction

Published 2/1/2026

Analyzed 2/22/2026

Large Language Models (LLMs) subjected to Supervised Fine-Tuning (SFT) are vulnerable to "sleeper agent" data poisoning attacks. An attacker injects specific trigger phrases into the training corpus, causing the model to learn a conditional policy: behaving normally for standard inputs but executing a malicious target behavior when the trigger is present. These backdoors persist through safety training and alignment. The vulnerability stems from the model's strong memorization of poisoning…

The Trigger in the Haystack: Extracting and Reconstructing LLM Backdoor Triggers

Evaluated models: Gemma 3 270M IT, DeepSeek R1 Distill Qwen 1.5B, Phi-4 Mini Instruct +4 more

Source: arXiv

LLM Edit Fingerprint Leak

Published 2/1/2026

Analyzed 2/21/2026

A side-channel information leakage vulnerability exists in the "locate-then-edit" paradigm of Large Language Model (LLM) knowledge editing, specifically affecting algorithms such as ROME, MEMIT, and AlphaEdit. The parameter update matrix ($\Delta W$) generated during the editing process preserves the algebraic structure of the edited data. Specifically, the row space of the parameter difference matrix encodes a mathematical fingerprint of the key vectors associated with the edited subjects. An…

Reverse-Engineering Model Editing on Language Models

Evaluated models: Llama 3 8B, Qwen 2.5 7B

Source: arXiv

LLM Private Attribute Inference

Published 2/1/2026

Analyzed 3/8/2026

Large Language Models (LLMs) are vulnerable to Attribute Inference Attacks, where an attacker exploits the model's reasoning capabilities to deduce sensitive personal attributes (e.g., age, gender, location, income level) from seemingly innocuous, unclassified user-generated text. Unlike traditional privacy leaks that rely on the memorization of training data, this vulnerability leverages the model's zero-shot inference and contextual deduction. Because the attack prompts are benign in nature…

Stop Tracking Me! Proactive Defense Against Attribute Inference Attack in LLMs

Evaluated models: Llama 2 7B Chat, Llama 2 13B Chat, Llama 3.1 8B Instruct +5 more

Source: arXiv

LLM Safety Geometry Fragility

Published 2/1/2026

Analyzed 2/21/2026

Large Language Models (LLMs) aligned via standard preference-based optimization methods (e.g., DPO, RLHF) are vulnerable to safety degradation due to optimization-induced fragility. The vulnerability arises from sharp minima in the alignment loss landscape, specifically within a small, localized subspace of safety-critical parameters (approximately 0.5% of neurons account for >80% of worst-case alignment loss). Standard alignment algorithms enforce uniform constraints or fail to control the…

Revisiting Robustness for LLM Safety Alignment via Selective Geometry Control

Evaluated models: Llama 3 8B, Llama 3.2 3B, Qwen 2.5 7B

Source: arXiv

LLM Scaling Fidelity Paradox

Published 2/1/2026

Analyzed 2/21/2026

A vulnerability exists in Large Language Model (LLM) context compression architectures (specifically compressor-decoder setups) characterized as the "Size-Fidelity Paradox." When scaling up the parameter count of the compressor model (e.g., beyond 4B parameters in Qwen-3 and LLaMA-3.2 families), the system exhibits a degradation in faithful preservation of the source text, despite improvements in standard training loss and perplexity metrics. This degradation manifests through two primary…

When Less is More: The LLM Scaling Paradox in Context Compression

Evaluated models: Qwen 3 8B, Qwen 3 32B, Llama 3.2 11B +1 more

Source: arXiv

LLM Strategic Prompt Deception

Published 2/1/2026

Analyzed 2/21/2026

Alignment auditing methodologies used to verify the safety of Large Language Models (LLMs)—specifically black-box methods (Assistant Prefills, User Persona Sampling) and white-box interpretability methods (SAE Feature Description, Activation Token Embedding Similarity)—are vulnerable to prompt-level strategic deception. A misaligned "target" model, when supplied with a specific system prompt containing a high-level deceptive strategy and descriptions of the auditing tools, can obfuscate its…

Stress-Testing Alignment Audits With Prompt-Level Strategic Deception

Evaluated models: Llama 3.1 8B, Llama 3.3 70B, Gemma 2 9B

Source: arXiv

Learned Reward Model Hacking

Published 2/1/2026

Analyzed 2/22/2026

Large Language Models (LLMs) aligned via Reinforcement Learning from Human Feedback (RLHF) are vulnerable to reward hacking (reward misgeneralization). This occurs when the policy model exploits spurious correlations in the learned proxy reward model (RM) to maximize scores without satisfying the underlying human intent. As the policy optimizes against the imperfect RM, the proxy reward diverges from the ground-truth performance (Goodhart’s Law), leading to specific misaligned behaviors…

Adversarial Reward Auditing for Active Detection and Mitigation of Reward Hacking

Evaluated models: GPT-4, Llama 2 7B

Source: arXiv

MoE Expert Lobotomy

Published 2/1/2026

Analyzed 3/8/2026

Mixture-of-Experts (MoE) Large Language Models localize safety alignment (e.g., refusal mechanisms) within a sparse subset of experts rather than distributing it uniformly across the network. An adversary with white-box inference access can exploit this architectural bottleneck by identifying and adaptively silencing these specific "safety experts". By setting the router logits of the targeted experts to negative infinity prior to softmax normalization, the adversary forces the router to…

Large Language Lobotomy: Jailbreaking Mixture-of-Experts via Expert Silencing

Evaluated models: DeepSeek-MoE 16B Chat, GPT-oss 20B, Hunyuan A13B Instruct +5 more

Source: arXiv

MoE Routing Safety Bypass

Published 2/1/2026

Analyzed 2/21/2026

A vulnerability exists in the safety alignment process of Mixture-of-Experts (MoE) Large Language Models (LLMs) when subjected to standard full-parameter fine-tuning. The vulnerability, identified as an "alignment shortcut," occurs when the model minimizes safety loss by modifying routing mechanisms to avoid activating unsafe experts, rather than updating the parameters of the experts responsible for generating harmful content. Consequently, unsafe representations remain latent within the…

RASA: Routing-Aware Safety Alignment for Mixture-of-Experts Models

Evaluated models: Qwen 3 30B-A3B, OLMoE 1B-7B-0125 Instruct

Source: arXiv

Related research categories