Architecture Components

Fine-tuning Research

Security research involving model fine-tuning processes

126 matching entries

Fine-tuning research results

126 entries

Boilerplate safety tuning causes false refusals on benign requests

Published Sep 4, 2026Analyzed Sep 9, 2026Source: arXiv

Controlled safety-tuning experiments link boilerplate refusal statements to unnecessary refusals of benign requests. Request-specific rationales improve benign compliance, with benchmark-dependent safety tradeoffs.

Refuse without Refusal: A Structural Analysis of Safety-Tuning Responses for Reducing False Refusals in Language Models

Evaluated models: Llama 3.1 8B, Mistral 7B v0.3, Gemma 2 9B +9 more

Selective refusal gaps in visual question answering

Published Sep 4, 2026Analyzed Sep 9, 2026Source: arXiv

KoNA measures whether vision-language models answer valid image questions while refusing unsafe components or correcting unsupported premises. Its 9,300 question-answer pairs include mixed and fully answerable controls.

Knowing What Not to Answer: Selective Non-Compliance in Vision-Language Models

Evaluated models: InternVL3 2B Instruct, InternVL3-78B-Instruct, Qwen 2.5 VL 3B Instruct +5 more

Backdoored PEFT adapters and limits of weight-only detection

Published Aug 3, 2026Analyzed Sep 9, 2026Source: arXiv

Third-party fine-tuning adapters may contain backdoors. Z-PEFT screens adapter weights using spectral features, evaluated on PADBench's 13,300 adapters.

Z-PEFT: Zero-shot Backdoor Detection in Parameter-Efficient Fine-Tuning via Canonical Spectral Signatures

Evaluated models: Llama 2 7B, Llama 2 13B, Qwen 1.5 7B +2 more

LLM Robot JSON Backdoor

Published Apr 4, 2026Analyzed Apr 10, 2026Source: arXiv

A supply-chain vulnerability in LLM-mediated robotic control systems allows attackers to execute unauthorized physical actions via structured backdoor attacks embedded in LoRA adapters. By poisoning the fine-tuning dataset to map specific natural-language trigger phrases directly to malicious, syntactically valid JSON control commands (structured-output poisoning), the backdoor bypasses natural-language reasoning layers and propagates deterministically to downstream robotic middleware (e.g…

From Prompt to Physical Action: Structured Backdoor Attacks on LLM-Mediated Robotic Control Systems

Evaluated models: Llama 3.1 8B Instruct, Gemma 2 9B IT, DeepSeek R1 Distill Llama 8B +2 more

Weight Orthogonalization Attack Empowerment

Published Apr 2, 2026Analyzed Apr 10, 2026Source: arXiv

An issue in large language models (LLMs) with white-box weight access allows attackers to permanently bypass safety guardrails via Weight Orthogonalization (WO). By calculating a model's "refusal vector"—the mean-difference vector between harmful and harmless instruction activations in the residual stream—an attacker can orthogonalize the model's weights to prevent it from writing to this refusal direction ($W^{\prime}\leftarrow W-rr^{\intercal}W$). Unlike jailbreak-tuning or data poisoning…

Understanding the Effects of Safety Unalignment on Large Language Models

Evaluated models: Qwen 3 4B Instruct 2507, Llama 3.1 8B Instruct, Qwen 2.5 14B +3 more

LLM Judge Manipulation

Published Mar 31, 2026Analyzed Apr 10, 2026Source: arXiv

Generative reward models deployed as LLM-as-a-Judge (LaaJ) evaluators contain a logic bypass vulnerability where superficial "master key" inputs trigger false positive rewards regardless of actual response quality. Instead of evaluating the candidate's output, large judge models are inadvertently triggered by specific token sequences to solve the prompt independently. This allows malicious actors or policy models undergoing reinforcement learning to consistently game the reward signal by…

Security in LLM-as-a-Judge: A Comprehensive SoK

Evaluated models: GPT-4o, o1, Qwen 2.5 72B Instruct +1 more

Zero-Tax Fine-Tune Bypass

Published Mar 30, 2026Analyzed Apr 10, 2026Source: arXiv

An adversarial fine-tuning vulnerability exists in LLMs protected by text-based safety classifiers (such as Anthropic's Constitutional Classifiers). By utilizing a two-stage curriculum learning combined with hybrid RL+SFT (GRPO), an attacker can fine-tune a model to communicate using a minimal substitution cipher (replacing only 7-8 high-frequency characters) disguised within benign technical templates (e.g., forensic logs with 0x prefixes). This "Trojan-Speak" methodology bypasses text-level…

Trojan-Speak: Bypassing Constitutional Classifiers with No Jailbreak Tax via Adversarial Finetuning

Evaluated models: Claude Haiku 4.5, Qwen 3 4B, Qwen 3 8B +2 more

Colluding LoRA Emergent Harm

Published Mar 13, 2026Analyzed Apr 10, 2026Source: arXiv

A compositional vulnerability in modular Large Language Models (LLMs) allows attackers to bypass safety alignment by distributing malicious weight updates across multiple Parameter-Efficient Fine-Tuning (PEFT) adapters (e.g., LoRA). The malicious adapters are anchored to valid functional subspaces (e.g., math, coding) and exhibit benign behavior when evaluated in isolation, successfully evading standard unit-centric safety scans and static weight-space defenses. However, when a user linearly…

Colluding LoRA: A Composite Attack on LLM Safety Alignment

Evaluated models: Llama 3 8B, Qwen 2.5 7B, Gemma 2 2B

LLM Steganographic Covert Channel

Published Mar 9, 2026Analyzed Apr 10, 2026Source: arXiv

A malicious finetuning vulnerability exists in Large Language Models (LLMs) that process zero-width Unicode characters. An attacker can bypass training-data moderation filters and inference-time safety guardrails by finetuning the model to decode and encode invisible-character steganography. By injecting target malicious interactions encoded in a base-4 representation of zero-width characters alongside benign plaintext cover text during supervised finetuning (SFT), the model learns to process…

Invisible Safety Threat: Malicious Finetuning for LLM via Steganography

Evaluated models: GPT-4.1, Llama 3.3 70B Instruct, Phi-4 +1 more

Discrete Token Subversion

Published Feb 20, 2026Analyzed Mar 9, 2026Source: arXiv

Discrete image tokenizers are vulnerable to unsupervised embedding-space adversarial attacks. Attackers can apply $\ell_p$-bounded perturbations to an input image to maximize the $\ell_2$ distance of the pre-quantization continuous embeddings produced by the tokenizer's vision encoder. This forces the vector quantizer to cross discrete cell boundaries and assign incorrect codebook vectors, fundamentally altering the resulting token sequence. Because the attack targets the pre-quantization…

On the Adversarial Robustness of Discrete Image Tokenizers

Evaluated models: Llama 2 7B

Related research categories