Attack Context

API Research

Security research involving model API implementations

83 matching entries

API research results

83 entries

Custom GPT Prompt Injection

Published 11/1/2023

Analyzed 12/28/2024

A prompt injection vulnerability in OpenAI's custom GPT models allows attackers to extract the system prompt and potentially leak user-uploaded files. Attackers craft malicious prompts that manipulate the LLM into revealing sensitive information, even when defensive prompts are in place. The vulnerability is exacerbated when the model includes a code interpreter.

Assessing prompt injection risks in 200+ custom gpts

Evaluated models: Not reported

Source: arXiv

GPT-4v System Prompt Leakage

Published 11/1/2023

Analyzed 12/29/2024

A system prompt leakage vulnerability in GPT-4V allows extraction of internal system prompts through carefully crafted, incomplete conversations combined with image input. Extracted prompts can be used as highly effective jailbreak prompts, bypassing safety restrictions and leading to undesirable outputs, including revealing personally identifiable information from images.

Jailbreaking gpt-4v via self-adversarial attacks with system prompts

Evaluated models: GPT-4, GPT-4V, LLaVA 1.5

Source: arXiv

Prompt-SQL Injection Vulnerability

Published 8/1/2023

Analyzed 1/26/2025

Large Language Model (LLM)-integrated web applications using Langchain (and potentially similar middleware) are vulnerable to Prompt-to-SQL (P2SQL) injection attacks. Unsanitized user prompts can be crafted to cause the LLM to generate malicious SQL queries, leading to unauthorized database access (read and write operations). This vulnerability bypasses attempts to restrict the LLM through prompt engineering alone.

From prompt injections to sql injection attacks: How protected is your llm-integrated web application?

Evaluated models: GPT-3.5 Turbo, GPT-4, PaLM 2 +1 more

Source: arXiv

Related research categories