Language Model Security Database

Multimodal Large Language Models (LLMs) processing speech input are vulnerable to adversarial attacks. Imperceptible perturbations added to audio input can cause the model to generate unsafe or harmful text responses, overriding built-in safety mechanisms. The attacks are effective even with limited knowledge of the model's internal workings, demonstrating transferability across different models.

Large Language Models (LLMs) are vulnerable to a novel prompting technique, "conditional Variational-autoencoder-Like Prompt" (VLPrompt), which enables the generation of highly convincing fake news articles. VLPrompt overcomes limitations of previous methods by eliminating the need for additional human-collected data while maintaining contextual coherence and detail. This allows for the automated mass-production of realistic-sounding fake news.

A color-aware attack, Self Color Testing-based Substitution (SCTS), bypasses watermarking mechanisms in LLMs designed to identify AI-generated text. SCTS exploits the LLM's compliance with instructions to infer the "color" (green/red token classification) of tokens, allowing for targeted substitution of watermarked tokens with non-watermarked tokens, thus evading watermark detection. The attack is particularly effective against watermarks that utilize logit perturbation to bias token selection.

A data poisoning attack, termed ImgTrojan, allows adversaries to bypass safety mechanisms in Vision-Language Models (VLMs) by injecting a small number of maliciously crafted image-caption pairs into the training dataset. These poisoned pairs associate seemingly benign images with jailbreak prompts, causing the VLM to generate unsafe outputs when presented with the poisoned images at inference time. The attack's success rate is notably high even with a very low poison ratio (e.g., one poisoned…

A novel adversarial suffix embedding translation framework (ASETF) enables efficient and highly successful attacks against large language models (LLMs). ASETF optimizes continuous adversarial suffix embeddings, then translates these embeddings into coherent, human-readable text. This bypasses existing defenses which rely on detecting unusual or nonsensical suffixes. The attack achieves a high success rate across multiple LLMs, including both open-source and black-box models.

A novel gradient-guided backdoor trigger learning (GBTL) algorithm allows adversaries to inject backdoor triggers into instruction-tuning datasets for Large Language Models (LLMs). These triggers, appended to the input content without altering the instruction or label, cause the LLM to generate a pre-determined malicious response during inference, even with minimal poisoned training data (e.g., 1%). The triggers maintain low perplexity, making them difficult to detect by standard filtering…

Large Language Models (LLMs) are vulnerable to black-box identity verification attacks using Targeted Random Adversarial Prompts (TRAP). TRAP leverages adversarial suffixes to elicit a pre-defined response from a target LLM, while other models produce random outputs, enabling identification of the specific LLM used within a third-party application via black-box access. This allows unauthorized identification of the underlying LLM even without access to model weights or internal parameters.

Multimodal Large Language Models (MLLMs) are vulnerable to a jailbreaking attack using crafted images (image Jailbreaking Prompts or imgJPs). These imgJPs, when presented as input alongside malicious prompts, cause the MLLM to bypass safety mechanisms and generate objectionable content, including instructions for harmful activities like identity theft or creation of violent video games. The attack demonstrates both prompt-universality (a single imgJP works across multiple prompts) and, to a…

Newly added APIs to large language models (LLMs), such as fine-tuning, function calling, and knowledge retrieval, introduce novel attack vectors that bypass existing safety mechanisms and enable various malicious activities. Specifically, fine-tuning with even a small number of carefully crafted examples can remove or weaken built-in safety guardrails, resulting in the generation of misinformation, disclosure of private information (PII), and the creation of malicious code. Function calling…

Large Language Models (LLMs) such as Llama 2 and Vicuna exhibit a vulnerability where specific layers (e.g., layer 3 in Llama2-13B, layer 1 in Llama2-7B and Vicuna-13B) overfit to harmful prompts, resulting in a disproportionate influence on the model's output for such prompts. This overfitting creates a narrow "safety" mechanism easily bypassed by adversarial prompts designed to avoid triggering these specific layers. Additionally, a single neuron (e.g., neuron 2100 in Llama2 and Vicuna)…