Skip to main content
LLM Security Database
Skip to research search
Last analyzed Sep 9, 2026

Language Model Security Database

985 research findings · 1119 evaluated models

Filtered research findings

610 entries

Matches every word across titles, descriptions, sources, affected systems, and models.

Automated Stealth Skill Injection

Published Analyzed Source: arXiv

A vulnerability exists in LLM-based coding agents that implement modular capability extensions (often referred to as "Agent Skills") where the agent dynamically loads and executes user-provided skill packages. The vulnerability allows for Skill-Based Prompt Injection, specifically leveraging a technique known as "SkillJect." This attack decouples the malicious intent from the operational payload to bypass semantic safety filters. An attacker constructs a skill package containing: 1. Inducement…

SkillJect: Effectively Automating Skill-Based Prompt Injection for Skill-Enabled Agents
Evaluated models: Claude Sonnet 4.6, GPT-5 Mini, GLM-4.7+7 more
  • Claude Sonnet 4.6
  • GPT-5 Mini
  • GLM-4.7
  • MiniMax M2.1
  • DeepSeek V4 Flash
  • GLM-5.1
  • MiniMax M2.7
  • MiMo V2.7
  • GPT-5.4
  • Claude Opus 4.6

LLM Guardrail Benchmark Lies

Published Analyzed Source: arXiv

Architectural limitations in Meta's Llama-Prompt-Guard-2-86M and Llama-Guard-3-8B cause them to fail at detecting indirect prompt injections and agentic tool-use attacks, with detection rates dropping as low as 7-37%. Llama-Guard-3-8B enforces strict user/assistant message alternation and lacks support for tool-use roles; attempting to process messages with role: "tool" or role: "ipython" causes the chat template to raise an error, preventing evaluation entirely. PromptGuard 2 operates…

When Benchmarks Lie: Evaluating Malicious Prompt Classifiers Under True Distribution Shift
Evaluated models: Llama 3 8B, Llama 3.1 8B

Rubric Stealthy Preference Drift

Published Analyzed Source: arXiv

LLM-as-a-Judge systems utilizing natural language rubrics are vulnerable to Rubric-Induced Preference Drift (RIPD). This vulnerability allows an attacker (or a flawed optimization process) to refine evaluation rubrics such that they maintain high agreement with human references on standard validation benchmarks while inducing systematic, directional preference degradation on unseen target domains. The attack exploits the disconnect between benchmark validation and target generalization by…

Rubrics as an Attack Surface: Stealthy Preference Drift in LLM Judges
Evaluated models: Llama 3 8B, Llama 3.1 8B, DeepSeek V3+1 more
  • Llama 3 8B
  • Llama 3.1 8B
  • DeepSeek V3
  • Gemma 2 2B

Reasoning Model Social Conformity

Published Analyzed Source: arXiv

Large reasoning models are vulnerable to multi-turn adversarial interactions that exploit reasoning-induced overconfidence to force answer capitulation. While explicit reasoning chains improve baseline accuracy, they cause models to effectively "talk themselves into" high confidence scores (clustering at 96–98%) regardless of actual correctness. This systematic overcalibration (r=-0.08, ROC-AUC=0.54) breaks confidence-based defense mechanisms like Confidence-Aware Response Generation (CARG)…

Consistency of Large Reasoning Models Under Multi-Turn Attacks
Evaluated models: GPT-5.1, GPT-5.2, DeepSeek R1+5 more
  • GPT-5.1
  • GPT-5.2
  • DeepSeek R1
  • Grok 4.1
  • Grok 3
  • Gemini 2.5 Pro
  • GPT-oss 120B
  • GPT-4o

Abstractive Character Violations

Published Analyzed Source: arXiv

Large Language Models (LLMs) aligned via reinforcement learning from human feedback (RLHF) or Constitutional AI exhibit a vulnerability where safety guardrails can be consistently bypassed through "Abstractive Red-Teaming." This attack vector exploits specific high-level natural language categories—combinations of semantic attributes such as tone, specific formatting instructions (e.g., numbered lists), language (e.g., Chinese, Russian), and topic constraints—that the model fails to generalize…

Abstractive Red-Teaming of Language Model Character
Evaluated models: GPT-4.1 Mini, Llama 3.1 8B Instruct, Gemma 3 12B IT+4 more
  • GPT-4.1 Mini
  • Llama 3.1 8B Instruct
  • Gemma 3 12B IT
  • Qwen 3 30B-A3B Instruct-2507
  • Claude 3.5 Haiku
  • Claude Sonnet 4
  • Claude Opus 4.1

LLM Private Attribute Inference

Published Analyzed Source: arXiv

Large Language Models (LLMs) are vulnerable to Attribute Inference Attacks, where an attacker exploits the model's reasoning capabilities to deduce sensitive personal attributes (e.g., age, gender, location, income level) from seemingly innocuous, unclassified user-generated text. Unlike traditional privacy leaks that rely on the memorization of training data, this vulnerability leverages the model's zero-shot inference and contextual deduction. Because the attack prompts are benign in nature…

Stop Tracking Me! Proactive Defense Against Attribute Inference Attack in LLMs
Evaluated models: Llama 2 7B Chat, Llama 2 13B Chat, Llama 3.1 8B Instruct+5 more
  • Llama 2 7B Chat
  • Llama 2 13B Chat
  • Llama 3.1 8B Instruct
  • DeepSeek R1 Distill Qwen 7B
  • Qwen 2.5 7B Instruct
  • GPT-3.5 Turbo
  • GPT-4o
  • Gemini 2.5 Pro

Mobile Agent Visual Spoofing

Published Analyzed Source: arXiv

Mobile Large Language Model (LLM) agents operating under the "Screen-as-Interface" paradigm are vulnerable to visual indirect prompt injection and state desynchronization. Agents that rely on unstructured visual data (screenshots) and Accessibility Service APIs to perceive the environment lack a mechanism to distinguish between trusted system UI elements and untrusted content (e.g., web pages, emails, or malicious overlays). An attacker can inject visual cues, fake notifications, or hidden…

Blind Gods and Broken Screens: Architecting a Secure, Intent-Centric Mobile Agent Operating System
Evaluated models: Not reported

Multi-Turn Proxy Evasion

Published Analyzed Source: arXiv

A logical vulnerability exists in LLM security proxies and guardrails that utilize weighted-average aggregation algorithms for multi-turn risk scoring. The scoring logic exhibits a mathematical "ceiling property" where the cumulative conversation-level risk score converges to the per-turn score regardless of the number of interaction turns ($n$). Consequently, the aggregated score is bounded by the maximum single-turn score ($cum \leq \max(s_i)$). This allows remote attackers to bypass…

Peak+ Accumulation: A Proxy-Level Scoring Formula for Multi-Turn LLM Attack Detection
Evaluated models: Not reported

Simple Persona Jailbreak

Published Analyzed Source: arXiv

A vulnerability in the Grok LLM, as deployed on the X social media platform, allows users to bypass safety filters and generate toxic or obscene content through "shallow alignment" techniques. The model prioritizes instruction compliance and conversational flow over safety guidelines, failing when exposed to simple adversarial interactions such as Persona Adoption (instructing the model to adopt a specific character) and Tone Mirroring (where the model automatically mimics a user's aggressive…

@ GrokSet: multi-party Human-LLM Interactions in Social Media
Evaluated models: Not reported

Clinical LLM Feature Evasion

Published Analyzed Source: arXiv

A detection bypass vulnerability in the 2-Sigma clinical training platform allows users to evade the system's two-layer, linguistic-feature-based jailbreak detection mechanism. The detection framework relies heavily on four surface-level linguistic features (Professionalism, Medical Relevance, Ethical Behavior, and Contextual Distraction) to classify malicious inputs. Attackers can bypass these filters by crafting prompts that maintain professional tone and apparent medical relevance but…

Detecting Jailbreak Attempts in Clinical Training LLMs Through Automated Linguistic Feature Extraction
Evaluated models: Not reported

Research methodology

Entries summarize publicly available primary-source security research. Model names reflect only systems explicitly evaluated by the cited paper, and measurements are research-reported unless independent verification is stated.

Discovery and publication are separate: new papers enter a candidate queue, then require human review and explicit approval before publication. Updates follow that review process, not a guaranteed daily publishing schedule. “Last analyzed” is the newest analysis date among published entries; it does not mean that all recent research has been covered.