Language Model Security Database

Large Language Models (LLMs) are vulnerable to a novel attack leveraging subconscious exploitation and echopraxia. Attackers craft prompts that subtly guide the LLM to echo malicious content it has implicitly learned during pre-training but is programmed to suppress. This bypasses safety mechanisms designed to prevent the generation of harmful content. The technique involves extracting malicious knowledge from the LLM's conditional probability distribution (representing its "subconscious") and…

A system prompt leakage vulnerability in GPT-4V allows extraction of internal system prompts through carefully crafted, incomplete conversations combined with image input. Extracted prompts can be used as highly effective jailbreak prompts, bypassing safety restrictions and leading to undesirable outputs, including revealing personally identifiable information from images.