Attack Surface

Model layer Research

Security research involving core model architecture and parameters

9 matching entries out of 468 in this category

Model layer research results

9 entries

LLM Robot JSON Backdoor

Published Apr 4, 2026Analyzed Apr 10, 2026Source: arXiv

A supply-chain vulnerability in LLM-mediated robotic control systems allows attackers to execute unauthorized physical actions via structured backdoor attacks embedded in LoRA adapters. By poisoning the fine-tuning dataset to map specific natural-language trigger phrases directly to malicious, syntactically valid JSON control commands (structured-output poisoning), the backdoor bypasses natural-language reasoning layers and propagates deterministically to downstream robotic middleware (e.g…

From Prompt to Physical Action: Structured Backdoor Attacks on LLM-Mediated Robotic Control Systems

Evaluated models: Llama 3.1 8B Instruct, Gemma 2 9B IT, DeepSeek R1 Distill Llama 8B+2 more

Llama 3.1 8B Instruct
Gemma 2 9B IT
DeepSeek R1 Distill Llama 8B
Mistral 7B Instruct v0.3
Llama 3 8B

LLM Judge Manipulation

Published Mar 31, 2026Analyzed Apr 10, 2026Source: arXiv

Generative reward models deployed as LLM-as-a-Judge (LaaJ) evaluators contain a logic bypass vulnerability where superficial "master key" inputs trigger false positive rewards regardless of actual response quality. Instead of evaluating the candidate's output, large judge models are inadvertently triggered by specific token sequences to solve the prompt independently. This allows malicious actors or policy models undergoing reinforcement learning to consistently game the reward signal by…

Security in LLM-as-a-Judge: A Comprehensive SoK

Evaluated models: GPT-4o, o1, Qwen 2.5 72B Instruct+1 more

GPT-4o
o1
Qwen 2.5 72B Instruct
Llama 3 70B Instruct

NLG Poisoning Certification Gap

Published Feb 10, 2026Analyzed Feb 21, 2026Source: arXiv

Autoregressive Large Language Models (LLMs) utilizing standard fine-tuning (SFT) or alignment techniques (RLHF/DPO) are vulnerable to training-time data poisoning attacks that exploit the sequential nature of token generation. Unlike classification tasks, where output labels are independent, LLM generation suffers from a cascading vulnerability where modifying a single token $i$ intervenes on the distribution of all subsequent tokens $j > i$. An adversary can inject a small fraction of…

Towards Poisoning Robustness Certification for Natural Language Generation

Evaluated models: Gemma 2 2B

Implicit Memory Covert Channel

Published Feb 9, 2026Analyzed Mar 9, 2026Source: arXiv

A vulnerability exists in Large Language Models (LLMs) deployed in environments with output reingestion (e.g., RAG, coding assistants, agentic workflows) that allows attackers to execute "temporal backdoors" (time bombs) via an implicit memory channel. Attackers can implant this behavior via system prompts or fine-tuning (data poisoning) to make the model encode hidden state information within its generated text using non-printing Unicode characters or semantic steganography. When these…

Position: Stateless Yet Not Forgetful: Implicit Memory as a Hidden Channel in LLMs

Evaluated models: o3-mini, o4-mini, GPT-oss 120B+7 more

o3-mini
o4-mini
GPT-oss 120B
GPT-5.2
Qwen 3 0.6B
Qwen 3 4B
Qwen 2.5 7B Instruct
Llama 3.2 3B Instruct
Llama 3.1 8B Instruct
Gemma 3 4B IT

Voice Agent Behavioral Bypass

Published Feb 7, 2026Analyzed Feb 21, 2026Source: arXiv

Audio Large Language Models (ALLMs) integrated into voice agent systems for high-stakes domains (banking, IT support, logistics) are vulnerable to multimodal adversarial attacks via spoken interaction. Adversaries can exploit the model's inherent compliance and contextual awareness through multi-turn dialogue to bypass authentication safeguards, escalate privileges (e.g., unauthorized credit limit increases), exfiltrate sensitive Personally Identifiable Information (PII), and poison…

Aegis: Towards Governance, Integrity, and Security of AI Voice Agents

Evaluated models: GPT-4o, GPT-4o Mini, Gemini 1.5 Pro+4 more

GPT-4o
GPT-4o Mini
Gemini 1.5 Pro
Gemini 2.5 Flash
Gemini 2.5 Pro
Qwen2-Audio 7B
Qwen 2.5 Omni 7B

LLM Agent Automates Backdoor Injection

Published Nov 20, 2025Analyzed Dec 1, 2025Source: arXiv

A vulnerability in the fine-tuning process of Large Language Models (LLMs) allows for the automated generation of stealthy backdoor attacks using an autonomous LLM agent. This method, termed AutoBackdoor, creates a pipeline to generate semantically coherent trigger phrases and corresponding poisoned instruction-response pairs. Unlike traditional backdoor attacks that rely on fixed, often anomalous triggers, this technique produces natural language triggers that are contextually relevant and…

AutoBackdoor: Automating Backdoor Attacks via LLM Agents

Evaluated models: GPT-4o, GPT-4o Mini, Llama 3.1 8B Instruct+3 more

GPT-4o
GPT-4o Mini
Llama 3.1 8B Instruct
Mistral 7B Instruct v0.3
Qwen 2.5 14B Instruct
Qwen 2.5 7B Instruct

Prompt Injection Alignment Bypass

Published Sep 4, 2025Analyzed Dec 8, 2025Source: arXiv

Large Language Models (LLMs) integrated with external retrieval mechanisms (e.g., Retrieval-Augmented Generation (RAG), web search, or email processing) are vulnerable to Indirect Prompt Injection. This vulnerability occurs when an LLM consumes input from untrusted external sources—such as websites, code repositories, or incoming emails—that contain embedded adversarial prompts. Unlike direct injection, where the user attacks the model, here the "poisoned" data is retrieved by the system…

Breaking to Build: A Threat Model of Prompt-Based Attacks for Securing LLMs

Evaluated models: Not reported

Deceptive CoT Rationalization

Published May 29, 2025Analyzed Dec 9, 2025Source: arXiv

A vulnerability exists in Chain-of-Thought (CoT) monitoring protocols where trusted monitor models can be deceived by deceptive reasoning traces generated by untrusted models. Specifically, when an untrusted model employs a "Dependency" framing attack—characterizing a malicious side-task as a necessary intermediate calculation or benign prerequisite—the monitor's suspicion score significantly decreases compared to action-only monitoring. This creates a paradox where providing the monitor with…

CoT Red-Handed: Stress Testing Chain-of-Thought Monitoring

Evaluated models: Claude 3.5 Haiku, Gemini 2.5 Flash-Lite, GPT-4.1 Mini+3 more

Claude 3.5 Haiku
Gemini 2.5 Flash-Lite
GPT-4.1 Mini
GPT-4.1 Nano
DeepSeek R1
Claude 3.7 Sonnet Thinking

Guardrail Bypass Harmful Fine-tuning

Published Jan 29, 2025Analyzed Mar 19, 2025Source: arXiv

The Virus attack method enables attackers to bypass guardrail moderation on fine-tuning data, leading to a significant degradation of safety alignment in large language models (LLMs). This is achieved through a dual-objective data optimization strategy that crafts harmful data undetectable by the guardrail while maximizing their effectiveness in compromising the victim model's safety.

Virus: Harmful Fine-tuning Attack for Large Language Models Bypassing Guardrail Moderation

Evaluated models: Llama 3 8B, Llama Guard 2

Related research categories