Architecture Components

Fine-tuning Research

Security research involving model fine-tuning processes

126 matching entries

Fine-tuning research results

126 entries

LLM Agent Automates Backdoor Injection

Published Nov 20, 2025Analyzed Dec 1, 2025Source: arXiv

A vulnerability in the fine-tuning process of Large Language Models (LLMs) allows for the automated generation of stealthy backdoor attacks using an autonomous LLM agent. This method, termed AutoBackdoor, creates a pipeline to generate semantically coherent trigger phrases and corresponding poisoned instruction-response pairs. Unlike traditional backdoor attacks that rely on fixed, often anomalous triggers, this technique produces natural language triggers that are contextually relevant and…

AutoBackdoor: Automating Backdoor Attacks via LLM Agents

Evaluated models: GPT-4o, GPT-4o Mini, Llama 3.1 8B Instruct+3 more

GPT-4o
GPT-4o Mini
Llama 3.1 8B Instruct
Mistral 7B Instruct v0.3
Qwen 2.5 14B Instruct
Qwen 2.5 7B Instruct

Self-Jailbreak Safety Override

Published Oct 24, 2025Analyzed Dec 9, 2025Source: arXiv

Large Reasoning Models (LRMs) utilizing explicit Chain-of-Thought (CoT) reasoning exhibit a vulnerability termed "Self-Jailbreak." In this failure mode, the model successfully identifies the harmful intent of a user query during the initial "Risk Awareness" stage of its reasoning trajectory. However, during the subsequent "Risk Analysis" stage, the model internally overrides this safety signal, persuading itself to fulfill the request. This override typically occurs through cognitive patterns…

When Models Outthink Their Safety: Unveiling and Mitigating Self-Jailbreak in Large Reasoning Models

Evaluated models: o1, DeepSeek R1, DeepSeek R1 0528+3 more

o1
DeepSeek R1
DeepSeek R1 0528
Qwen 3 8B
Qwen 3 14B
Qwen 3 32B

Benign Reasoning Self-Jailbreaks LLMs

Published Oct 23, 2025Analyzed Dec 8, 2025Source: arXiv

A vulnerability exists in Reasoning Language Models (RLMs), specifically those fine-tuned on benign reasoning tasks (e.g., mathematics, coding) such as DeepSeek-R1-distilled, s1.1, and Phi-4-mini-reasoning. The vulnerability, termed "Self-Jailbreaking," allows the model to circumvent its own safety guardrails during the internal Chain-of-Thought (CoT) generation process. Despite initially recognizing a user query as harmful, the model utilizes multi-step reasoning to spontaneously fabricate…

Self-Jailbreaking: Language Models Can Reason Themselves Out of Safety Alignment After Benign Reasoning Training

Evaluated models: s1.1 7B, Qwen 2.5 7B Instruct, DeepSeek R1 Distill Qwen 1.5B+6 more

s1.1 7B
Qwen 2.5 7B Instruct
DeepSeek R1 Distill Qwen 1.5B
DeepSeek R1 Distill Qwen 7B
DeepSeek R1 Distill Qwen 14B
DeepSeek R1 Distill Qwen 32B
DeepSeek R1 Distill Llama 8B
Phi-4 Mini Reasoning
Nemotron Research Reasoning

Leaked Bits Collapse Attack Queries

Published Oct 19, 2025Analyzed Dec 9, 2025Source: arXiv

Large Language Models (LLMs), specifically variants of GPT-4o, DeepSeek-R1, OLMo-2, and Llama-4, are vulnerable to accelerated adaptive adversarial attacks due to excessive information leakage in observable output signals. When these models expose "thinking processes" (Chain-of-Thought traces) or token-level log-probabilities (logits) to the end user, they leak significant mutual information $I(Z;T)$ regarding the model's safety state or hidden instructions. This leakage allows adaptive attack…

Bits Leaked per Query: Information-Theoretic Bounds on Adversarial Attacks against LLMs

Evaluated models: DeepSeek R1, GPT-4o Mini 2024-07-18, Llama 4 Maverick 17B+4 more

DeepSeek R1
GPT-4o Mini 2024-07-18
Llama 4 Maverick 17B
Llama 4 Scout 17B
OLMo 2 7B-1124
OLMo 2 13B-1124
OLMo 2 32B-0325

Agent Harassment Escalation

Published Oct 16, 2025Analyzed Feb 21, 2026Source: arXiv

Large Language Model (LLM) agents powered by LLaMA-3.1-8B-Instruct and Gemini-2.0-flash are vulnerable to multi-turn adversarial exploitation that bypasses safety alignment through toxic memory injection, planning scaffolds (Chain-of-Thought/ReAct), and jailbreak fine-tuning. Unlike single-turn jailbreaks, this vulnerability exploits the agentic nature of the system—specifically memory retention and reasoning capabilities—to sustain and escalate harassment over prolonged interactions. When…

Echoes of Human Malice in Agents: Benchmarking LLMs for Multi-Turn Online Harassment Attacks

Evaluated models: Llama 3.1 8B Instruct, Gemini 2.0 Flash 001

Special Token Jailbreak

Published Oct 11, 2025Analyzed Oct 31, 2025Source: arXiv

Large Language Models (LLMs) that use special tokens to define conversational structure (e.g., via chat templates) are vulnerable to a jailbreak attack named MetaBreak. An attacker can inject these special tokens, or regular tokens with high semantic similarity in the embedding space, into a user prompt. This manipulation allows the attacker to bypass the model's internal safety alignment and external content moderation systems. The attack leverages four primitives: 1. Response Injection…

MetaBreak: Jailbreaking Online LLM Services via Special Token Manipulation

Evaluated models: Claude Opus 4, Gemma 2 27B IT, GPT-4.1+9 more

Claude Opus 4
Gemma 2 27B IT
GPT-4.1
Llama 3.1 405B
Llama 3.1 8B
Llama 3.3 70B Instruct
Llama Guard
Llama Guard 3 8B
Phi-4
Prompt Guard
Qwen 2.5 72B Instruct
ShieldGemma 2 27B

Overfitting-induced Benign Jailbreak

Published Oct 3, 2025Analyzed Oct 13, 2025Source: arXiv

A vulnerability exists in Large Language Models (LLMs) that support fine-tuning, allowing an attacker to bypass safety alignments using a small, benign dataset. The attack, "Attack via Overfitting," is a two-stage process. In Stage 1, the model is fine-tuned on a small set of benign questions (e.g., 10) paired with identical, repetitive refusal answers. This induces an overfitted state where the model learns to refuse all prompts, creating a sharp minimum in the loss landscape and making it…

Attack via Overfitting: 10-shot Benign Fine-tuning to Jailbreak LLMs

Evaluated models: DeepSeek R1 Distill Llama 8B, GPT-3.5 Turbo, GPT-4.1+7 more

DeepSeek R1 Distill Llama 8B
GPT-3.5 Turbo
GPT-4.1
GPT-4.1 Mini
GPT-4o
GPT-4o Mini
Llama 2 7B Chat
Llama 3 8B Instruct
Qwen 2.5 7B Instruct
Qwen 3 8B

Black-Box Fine-Tuning Evasion

Published Oct 1, 2025Analyzed Jan 14, 2026Source: arXiv

Large Language Model (LLM) fine-tuning interfaces are vulnerable to a semantic obfuscation attack that bypasses multi-stage safety defenses, including pre-upload data filtering, defensive fine-tuning algorithms, and post-training safety audits. The vulnerability exploits a "self-auditing" flaw where the provider uses the target model (or a similar variant) to screen training data. Attackers can submit a small dataset (approx. 500 samples) where harmful answers are obfuscated using a…

Fine-Tuning Jailbreaks under Highly Constrained Black-Box Settings: A Three-Pronged Approach

Evaluated models: GPT-4o, GPT-4.1, GPT-4o Mini+5 more

GPT-4o
GPT-4.1
GPT-4o Mini
GPT-4.1 Mini
Llama 2 7B Chat
Gemma 1.1 7B IT
Qwen 2.5 7B Instruct
Claude Sonnet 4

Intermediate Denoising Priming

Published Oct 1, 2025Analyzed Dec 30, 2025Source: arXiv

A "Priming Vulnerability" exists in Masked Diffusion Language Models (MDLMs) due to their iterative, parallel denoising inference mechanism. Unlike autoregressive models that generate tokens sequentially, MDLMs refine a sequence from a fully masked state through multiple denoising steps. The vulnerability arises because standard safety alignment (e.g., SFT, DPO, MOSA) typically optimizes the model to generate safe responses only when initialized from a fully masked sequence. If an affirmative…

Toward Safer Diffusion Language Models: Discovery and Mitigation of Priming Vulnerability

Evaluated models: Not reported

LlamaGuard Obfuscation Bypass

Published Sep 21, 2025Analyzed Dec 8, 2025Source: arXiv

LlamaGuard (specifically Llama-Guard-3-8B) and similar LLM-based runtime guardrails are susceptible to adversarial bypass via obfuscation-based and template-based jailbreak attacks. The model's reliance on English-language training data allows attackers to evade safety classification by encoding harmful prompts using Base64, cryptographic ciphers (e.g., Caesar Cipher), or translating them into low-resource languages (e.g., Zulu). Furthermore, the model lacks sufficient alignment against…

DecipherGuard: Understanding and Deciphering Jailbreak Prompts for a Safer Deployment of Intelligent Software Systems

Evaluated models: Llama 3 8B

Related research categories