Attack Surface

Model layer Research

Security research involving core model architecture and parameters

7 matching entries out of 468 in this category

Model layer research results

7 entries

LLM Judge Manipulation

Published Mar 31, 2026Analyzed Apr 10, 2026Source: arXiv

Generative reward models deployed as LLM-as-a-Judge (LaaJ) evaluators contain a logic bypass vulnerability where superficial "master key" inputs trigger false positive rewards regardless of actual response quality. Instead of evaluating the candidate's output, large judge models are inadvertently triggered by specific token sequences to solve the prompt independently. This allows malicious actors or policy models undergoing reinforcement learning to consistently game the reward signal by…

Security in LLM-as-a-Judge: A Comprehensive SoK

Evaluated models: GPT-4o, o1, Qwen 2.5 72B Instruct+1 more

GPT-4o
o1
Qwen 2.5 72B Instruct
Llama 3 70B Instruct

Safety Steering Amplifies Jailbreaks

Published Feb 5, 2026Analyzed Feb 21, 2026Source: arXiv

Inference-time intervention techniques (also known as activation steering or model steering), utilized to adjust Large Language Model (LLM) behavior without retraining, contain a vulnerability related to robust specificity. When these methods are applied to reduce "over-refusal" (increasing compliance on benign but sensitive-sounding queries), they inadvertently degrade the model's adversarial robustness. Specifically, steering vectors derived from methods such as Difference-in-Means…

Steering Safely or Off a Cliff? Rethinking Specificity and Robustness in Inference-Time Interventions

Evaluated models: Llama 3.1 8B, Llama 3.2 3B, Qwen 2.5 7B+1 more

Llama 3.1 8B
Llama 3.2 3B
Qwen 2.5 7B
Gemma 2 2B

Agent-as-a-Proxy Monitor Bypass

Published Feb 4, 2026Analyzed Feb 21, 2026Source: arXiv

Hybrid monitoring protocols for Large Language Model (LLM) agents, specifically those that strictly evaluate both Chain-of-Thought (CoT) reasoning and tool usage (such as LlamaFirewall's AlignmentCheck and Extract-and-Evaluate monitors), are vulnerable to "Agent-as-a-Proxy" indirect prompt injection attacks. Unlike standard injections that target the agent, this attack targets the monitoring model itself. By embedding a malicious string—optimized via an algorithm named Parallel-GCG—into…

Bypassing AI Control Protocols via Agent-as-a-Proxy Attacks

Evaluated models: GPT-4o, Llama 3.1 8B, Mistral 7B+1 more

GPT-4o
Llama 3.1 8B
Mistral 7B
Qwen 2.5 7B

Unified Robustness Gap

Published Jan 31, 2026Analyzed Mar 9, 2026Source: arXiv

Text scoring models, including dense retrievers, rerankers, and reward models, are vulnerable to score manipulation attacks via search-based discrete perturbations and content injection. An attacker can systematically modify candidate texts using rudimentary string manipulations, gradient-guided token swaps (e.g., HotFlip), masked language modeling (MLM) swaps, or query/sentence injections to spuriously increase model scores. This structural failure condition allows an irrelevant passage or a…

Unifying Adversarial Robustness and Training Across Text Scoring Models

Evaluated models: E5 BERT-base, Qwen 3 0.6B, Llama 3.2 3B Instruct+2 more

E5 BERT-base
Qwen 3 0.6B
Llama 3.2 3B Instruct
Llama 3.1 8B Instruct
Skywork Reward V2

Single Word Rank Promotion

Published Jan 28, 2026Analyzed Feb 22, 2026Source: arXiv

Neural Ranking Models (NRMs) utilizing Transformer architectures (specifically BERT and T5-based re-rankers) are vulnerable to minimal adversarial perturbations that artificially promote a target document's rank. The vulnerability allows an attacker to manipulate ranking outcomes by inserting or substituting a single "query center" token—a word identified as the semantic centroid of the user's query—into the target document. The attack exploits the model's sensitivity to specific semantic…

One Word is Enough: Minimal Adversarial Perturbations for Neural Text Ranking

Evaluated models: Not reported

Prompt Injection Alignment Bypass

Published Sep 4, 2025Analyzed Dec 8, 2025Source: arXiv

Large Language Models (LLMs) integrated with external retrieval mechanisms (e.g., Retrieval-Augmented Generation (RAG), web search, or email processing) are vulnerable to Indirect Prompt Injection. This vulnerability occurs when an LLM consumes input from untrusted external sources—such as websites, code repositories, or incoming emails—that contain embedded adversarial prompts. Unlike direct injection, where the user attacks the model, here the "poisoned" data is retrieved by the system…

Breaking to Build: A Threat Model of Prompt-Based Attacks for Securing LLMs

Evaluated models: Not reported

Embedding Inversion Privacy Leak

Published Mar 17, 2025Analyzed Dec 30, 2025Source: arXiv

Retrieval-Augmented Generation (RAG) systems employing standard dense embedding models (e.g., Sentence-T5, SimCSE-BERT, RoBERTa, MPNet) for End-Cloud collaboration are vulnerable to Embedding Inversion Attacks (EIA). While embeddings are vector representations designed to be human-unrecognizable, they retain sufficient semantic information to allow an attacker with access to the vectors (e.g., a malicious or compromised cloud provider) to reconstruct the original sensitive plaintext input.

Safeguarding LLM Embeddings in End-Cloud Collaboration via Entropy-Driven Perturbation

Evaluated models: Not reported

Related research categories