Large Language Models (LLMs) are vulnerable to a novel adversarial attack, Alphabet Index Mapping (AIM), which achieves high success rates in bypassing safety filters ("jailbreaking"). AIM encodes prompts by converting characters to their alphabet indices, maximizing semantic dissimilarity while maintaining straightforward decoding instructions. This allows malicious prompts to evade detection based on semantic similarity, even when the LLM correctly decodes the intent.
Alphabet Index Mapping: Jailbreaking LLMs through Semantic Dissimilarity
Large Language Models (LLMs), specifically Transformer-based architectures, are vulnerable to an attention hijacking attack via optimized adversarial suffixes. The vulnerability resides in the shallow information flow mechanism of the attention layer, where specific token sequences (adversarial suffixes) can exert irregular and extreme dominance over the internal representation of the final chat template tokens immediately preceding generation. This "hijacking" suppresses the representation of…
Universal Jailbreak Suffixes Are Strong Attention Hijackers
Multiple Large Language Models (LLMs), including GPT-4o, Claude 3.5 Sonnet, Gemini 2.5 Flash, Gemma-2 27B, Gemma-3, and Mistral-Small-24B, exhibit a vulnerability where safety guardrails against demographic bias are bypassed through realistic contextual prompting. While prompt-based mitigations effectively suppress bias in simplified, controlled benchmarks, the introduction of realistic hiring contexts—specifically the combination of company culture descriptions (sourced from public career…
Robustly Improving LLM Fairness in Realistic Settings via Interpretability
Evaluated models: GPT-4o, Claude 3.5 Sonnet, Claude Sonnet 4+5 more
Alibaba Cloud PAI-Judge and PAI-Judge-Plus are vulnerable to a composite adversarial attack that exploits attention mechanism limitations in Large Language Models (LLMs). An authenticated attacker can manipulate automated evaluation outcomes by appending a long, irrelevant text suffix (approximately 1000 to 2000+ characters) to a response containing adversarial perturbations. This "long-suffix" strategy overwhelms the judge model's context window, causing the attention mechanism to degrade and…
LLMs Cannot Reliably Judge (Yet?): A Comprehensive Assessment on the Robustness of LLM-as-a-Judge
Evaluated models: GPT-4o, Llama 3.1 8B, Llama 3.3 70B+3 more
Large Language Model (LLM) agents capable of invoking external APIs are vulnerable to intent integrity violations. When an agent receives natural language instructions that are ambiguous, underspecified, or contain values not supported by the underlying API schema, the agent frequently fails to preserve user intent. Instead of rejecting the request or asking for clarification, the model may hallucinate parameter values, map unsupported requests to unsafe defaults, or execute actions on…
TAI3: Testing Agent Integrity in Interpreting User Intent
Mainstream Large Language Models (LLMs), including DeepSeek-R1, GPT-4o, Llama-3.3-70B-Instruct, and Qwen-Coder, are vulnerable to black-box jailbreak attacks that bypass safety alignment mechanisms to generate functional malicious code. The vulnerability is exploited through specific prompt engineering techniques, most notably "Benign Expression" (substituting malicious keywords with harmless synonyms) and "DRA" (Decomposed Requirement Attack), which conceal malicious intent within seemingly…
LLMs Caught in the Crossfire: Malware Requests and Jailbreak Challenges
Evaluated models: Claude 3.5 Sonnet 20240620, GPT-4o Preview 20240801, GPT-4o Mini 2024-07-18+26 more
A vulnerability exists in the safety alignment mechanisms of Large Language Models (LLMs) (including GPT-4, Claude 3, Gemini, and Qwen families) leading to "Implicit Harm." Unlike traditional jailbreaks that use overtly harmful queries, this vulnerability allows remote attackers to coerce the model into providing factually incorrect, plausible, and dangerous responses to benign-looking inputs. By employing "JailFlip" techniques—specifically constructed affirmative-type or denial-type queries…
Beyond Jailbreaks: Revealing Stealthier and Broader LLM Security Risks Stemming from Alignment Failures
Evaluated models: GPT-4.1, GPT-4.1 Mini, GPT-4o+3 more
Large Language Models (LLMs) deployed in automated peer review workflows are vulnerable to targeted textual adversarial attacks. By employing a technique defined as "Attack Focus Localization," an attacker can identify critical document segments via Longest Common Subsequence (LCS) matching between the original text and an initial LLM-generated review. Injecting semantic-preserving perturbations—such as character-level noise, synonym substitution (e.g., TextFooler), or stylistic transfer…
Breaking the Reviewer: Assessing the Vulnerability of Large Language Models in Automated Peer Review Under Textual Adversarial Attacks
Evaluated models: GPT-4o, Llama 3.3 70B, Mistral Large
Large Language Models (LLMs) utilized for code generation exhibit a vulnerability termed "Chain-of-Code Collapse" (CoCC), where the models fail to generate correct code when presented with semantically faithful but adversarially structured prompts. By applying transformations such as domain shifting (renaming variables/contexts), adding distracting constraints (irrelevant but plausible rules), or inverting objectives (negation), an attacker can cause the model to produce functionally incorrect…
Chain-of-Code Collapse: Reasoning Failures in LLMs via Adversarial Prompting in Code Generation
Evaluated models: Gemini 2.5 Flash Preview, Gemini 2.0 Flash, Claude 3.7 Sonnet+5 more
Large Language Models (LLMs), including Llama-3, Gemma-2, and Qwen2.5, are vulnerable to automated adversarial attacks generated via a Quality-Diversity Red-Teaming (QDRT) framework. This vulnerability arises from the models' inability to robustly defend against attackers trained via behavior-conditioned reinforcement learning that optimize for specific "goal-driven" behaviors. Unlike standard attacks that optimize solely for toxicity, QDRT trains a population of specialized attacker models to…
Quality-Diversity Red-Teaming: Automated Generation of High-Quality and Diverse Attackers for Large Language Models