Published 5/1/2023
Analyzed 12/29/2024
Large Language Models (LLMs), specifically ChatGPT versions 3.5 and 4.0, are vulnerable to prompt engineering attacks that circumvent built-in content restrictions. Attackers can craft malicious prompts, categorized into "pretending," "attention shifting," and "privilege escalation" techniques, to elicit responses containing prohibited content (e.g., instructions for illegal activities, generation of harmful content). The vulnerability stems from the LLM's inability to reliably distinguish…
Jailbreaking chatgpt via prompt engineering: An empirical study
Evaluated models: GPT-3.5 Turbo, GPT-4
Source: arXiv