Attack Surface

Model layer Research

Security research involving core model architecture and parameters

468 matching entries

Model layer research results

468 entries

Hidden Contextual Bias

Published 6/1/2025

Analyzed 12/30/2025

Multiple Large Language Models (LLMs), including GPT-4o, Claude 3.5 Sonnet, Gemini 2.5 Flash, Gemma-2 27B, Gemma-3, and Mistral-Small-24B, exhibit a vulnerability where safety guardrails against demographic bias are bypassed through realistic contextual prompting. While prompt-based mitigations effectively suppress bias in simplified, controlled benchmarks, the introduction of realistic hiring contexts—specifically the combination of company culture descriptions (sourced from public career…

Robustly Improving LLM Fairness in Realistic Settings via Interpretability

Evaluated models: GPT-4o, Claude 3.5 Sonnet, Claude Sonnet 4 +5 more

Source: arXiv

Hybrid LLM Jailbreak Strategy

Published 6/1/2025

Analyzed 7/14/2025

A hybrid jailbreak attack, combining gradient-guided token optimization (GCG) with iterative prompt refinement (PAIR or WordGame+), bypasses LLM safety mechanisms resulting in the generation of disallowed content. The hybrid approach leverages the strengths of both techniques, circumventing defenses effective against single-mode attacks. Specifically, the combination of semantically crafted prompts and strategically placed adversarial tokens confuse and overwhelm existing defenses.

Advancing Jailbreak Strategies: A Hybrid Approach to Exploiting LLM Vulnerabilities and Bypassing Modern Defenses

Evaluated models: Llama 2 7B, Llama Guard 2 8B, Mistral 7B +1 more

Source: arXiv

JailFlip Implicit Harm

Published 6/1/2025

Analyzed 12/8/2025

A vulnerability exists in the safety alignment mechanisms of Large Language Models (LLMs) (including GPT-4, Claude 3, Gemini, and Qwen families) leading to "Implicit Harm." Unlike traditional jailbreaks that use overtly harmful queries, this vulnerability allows remote attackers to coerce the model into providing factually incorrect, plausible, and dangerous responses to benign-looking inputs. By employing "JailFlip" techniques—specifically constructed affirmative-type or denial-type queries…

Beyond Jailbreaks: Revealing Stealthier and Broader LLM Security Risks Stemming from Alignment Failures

Evaluated models: GPT-4.1, GPT-4.1 Mini, GPT-4o +3 more

Source: arXiv

LLM Judge Subversion

Published 6/1/2025

Analyzed 12/9/2025

Alibaba Cloud PAI-Judge and PAI-Judge-Plus are vulnerable to a composite adversarial attack that exploits attention mechanism limitations in Large Language Models (LLMs). An authenticated attacker can manipulate automated evaluation outcomes by appending a long, irrelevant text suffix (approximately 1000 to 2000+ characters) to a response containing adversarial perturbations. This "long-suffix" strategy overwhelms the judge model's context window, causing the attention mechanism to degrade and…

LLMs Cannot Reliably Judge (Yet?): A Comprehensive Assessment on the Robustness of LLM-as-a-Judge

Evaluated models: GPT-4o, Llama 3.1 8B, Llama 3.3 70B +3 more

Source: arXiv

Semantic Prompt Distortion

Published 6/1/2025

Analyzed 12/9/2025

The Adaptive Greedy Binary Search (AGBS) framework exposes a vulnerability in Large Language Models (LLMs) regarding their susceptibility to semantic-preserving adversarial attacks. The vulnerability is exploited through a hierarchical decomposition strategy that identifies key semantic units (clauses and keywords) within a prompt. AGBS utilizes a dynamic threshold mechanism to adjust semantic similarity bounds in real-time during a beam search process, replacing tokens with candidates that…

Semantic-Preserving Prompt Hijacking: A Black-Box Adversarial Attack on Auto-Prompt Optimization

Evaluated models: GPT-3.5 Turbo, GPT-4 Turbo, GPT-4o +7 more

Source: arXiv

Stealthy Unlearning Degradation

Published 6/1/2025

Analyzed 6/30/2025

A vulnerability in fine-tuning-based large language model (LLM) unlearning allows malicious actors to craft manipulated forgetting requests. By subtly increasing the frequency of common benign tokens within the forgetting data, the attacker can cause the unlearned model to exhibit unintended unlearning behaviors when these benign tokens appear in normal user prompts, leading to a degradation of model utility for legitimate users. This occurs because existing unlearning methods fail to…

Keeping an eye on llm unlearning: The hidden risk and remedy

Evaluated models: Llama 3.1 8B, Mistral 7B v0.3

Source: arXiv

TeleLLM Safety Degradation

Published 6/1/2025

Analyzed 12/9/2025

Safety alignment degradation occurs in Large Language Models (LLMs) such as Llama-2, Llama-3, and Qwen-2 when subjected to Supervised Fine-Tuning (SFT) or Continual Pre-Training (CPT) on telecommunications domain datasets (TeleQnA, TeleData, TSpecLLM). The vulnerability arises because benign telecom data—characterized by structured tabular entries, long standardization reports, and complex mathematical formulas—shares gradient update directions with harmful data types. This results in…

SafeCOMM: What about Safety Alignment in Fine-Tuned Telecom Large Language Models?

Evaluated models: Llama 2 7B, Llama 3 8B, Llama 3.1 8B +2 more

Source: arXiv

Twin Prompt Jailbreak

Published 6/1/2025

Analyzed 7/14/2025

A white-box vulnerability allows attackers with full model access to bypass LLM safety alignments by identifying and pruning parameters responsible for rejecting harmful prompts. The attack leverages a novel "twin prompt" technique to differentiate safety-related parameters from those essential for model utility, performing fine-grained pruning with minimal impact on overall model functionality.

TwinBreak: Jailbreaking LLM Security Alignments based on Twin Prompts

Evaluated models: DeepSeek 7B, Gemma 2 27B, Gemma 2 2B +13 more

Source: arXiv

Universal Suffix Attention Hijack

Published 6/1/2025

Analyzed 12/30/2025

Large Language Models (LLMs), specifically Transformer-based architectures, are vulnerable to an attention hijacking attack via optimized adversarial suffixes. The vulnerability resides in the shallow information flow mechanism of the attention layer, where specific token sequences (adversarial suffixes) can exert irregular and extreme dominance over the internal representation of the final chat template tokens immediately preceding generation. This "hijacking" suppresses the representation of…

Universal Jailbreak Suffixes Are Strong Attention Hijackers

Evaluated models: Gemma 2 2B IT, Qwen 2.5 0.5B Instruct, Qwen 2.5 1.5B Instruct +2 more

Source: arXiv

Adversarial Suffix Jailbreak

Published 5/1/2025

Analyzed 12/30/2025

Large Language Models (LLMs), specifically instruction-tuned variants, are vulnerable to safety guardrail bypass via adversarial suffix injection. By appending a specific sequence of tokens—often semantically meaningless characters or carefully crafted distractors—to a malicious query, an attacker can manipulate the model's internal representation to override alignment training (RLHF). This coercion causes the model to affirmatively respond to otherwise refused requests, such as generating…

Adversarial Suffix Filtering: a Defense Pipeline for LLMs

Evaluated models: GPT-3.5, GPT-4o, Llama 2 7B +2 more

Source: arXiv

Related research categories