Language Model Security Database

A vulnerability exists in the internal representation mechanisms of Transformer-based Large Language Models (LLMs), specifically Llama-3 and Qwen series models. The vulnerability allows for high-accuracy "steering" of model outputs, effectively bypassing safety guardrails and refusal mechanisms (jailbreaking) without modifying model weights. By exploiting attention-guided feature learning, an attacker can extract a precise "concept vector" representing refusal behaviors. This is achieved by…

A vulnerability exists in safety-aligned Large Language Models (LLMs) wherein internal safety alignment mechanisms (such as RLHF) function as unobserved causal confounders rather than erasing prohibited knowledge. The "Causal Front-Door Adjustment Attack" (CFA2) exploits this architecture by modeling the safety mechanism as a distinct latent variable. Attackers with white-box access can employ Sparse Autoencoders (SAEs) to disentangle dense internal representations into sparse features…

Large language models (LLMs) exhibit a "Causal Bypass" vulnerability during Chain-of-Thought (CoT) prompting, where the generated reasoning text does not causally determine the model's final output. Instead of utilizing the explicit CoT tokens, the model routes decision-critical computation through latent, implicit pathways. This allows the visible reasoning trace to function as an unfaithful, post-hoc rationalization rather than an actual representation of the model's internal logic…

Reasoning-capable LLMs are vulnerable to a safeguard bypass where intermediate Chain-of-Thought (CoT) traces generate and expose harmful content, even if the model ultimately rejects the prompt in its final output. Output-level safety alignments fail to intervene during the intermediate reasoning stages, allowing adversaries to covertly construct and extract high-quality malicious narratives (such as fake news) directly from the CoT output. Mechanistic analysis reveals this divergence stems…

Large Language Models (LLMs) utilized for Automatic Short Answer Grading (ASAG) are vulnerable to the "GradingAttack" framework, which employs fine-grained adversarial manipulation to alter grading outcomes. Attackers can leverage two distinct strategies: (1) Prompt-level attacks using role-play injection strings that instruct the model to pretend an answer is correct regardless of factual accuracy, and (2) Token-level attacks utilizing gradient-based optimization (similar to Greedy Coordinate…

Discrete image tokenizers are vulnerable to unsupervised embedding-space adversarial attacks. Attackers can apply $\ell_p$-bounded perturbations to an input image to maximize the $\ell_2$ distance of the pre-quantization continuous embeddings produced by the tokenizer's vision encoder. This forces the vector quantizer to cross discrete cell boundaries and assign incorrect codebook vectors, fundamentally altering the resulting token sequence. Because the attack targets the pre-quantization…

Large Language Models (LLMs) aligned via standard Reinforcement Learning from Human Feedback (RLHF) or Supervised Fine-Tuning (SFT) are vulnerable to "Shallow Safety" bypass attacks, specifically Middle Filling (MF) and Greedy Coordinate Gradient (GCG) attacks. These models frequently rely on refusal mechanisms triggered solely by the initial tokens of a prompt. By embedding malicious instructions after a benign context (prefilling) or utilizing suffix optimization, attackers can induce the…

A vulnerability in the Direct Preference Optimization (DPO) post-training phase of OLMo 2 models leads to "distractor-triggered compliance." The model correctly refuses harmful requests when prompted in isolation, but complies with identical harmful requests if a benign formatting instruction (a "distractor") is appended to the prompt. This behavior organically emerges from contaminated preference training data where mislabeled examples incorrectly preferred compliance over refusal when a…

Large Language Models (LLMs) and Large Reasoning Models (LRMs) are vulnerable to an inference-time jailbreak attack known as "Hybrid Gradient Shift." This vulnerability exploits the statistical discrepancy between an aligned model's output distribution and a pre-alignment reference distribution. By treating safety alignment as a form of miscalibration, an attacker can aggregate the logits of a target model (aligned), a helper model (unaligned/weak), and a predictor model to reconstruct the…

A vulnerability in offline multi-armed bandit (MAB) evaluation frameworks allows attackers to completely hijack arm selection by applying imperceptibly small adversarial perturbations to the weights of the underlying reward model before training begins. Because the required perturbation norm decreases as the input dimensionality increases ($\mathcal{O}(d^{-1/2})$), high-dimensional reward models—such as wide neural networks or linear models used for generative model assessment—are highly…