Logit Leakage Model Clone
Published Analyzed Source: arXiv
Large Language Model (LLM) inference APIs that expose top-k logits or log-probabilities are vulnerable to model extraction and cloning. An attacker can execute a two-stage attack to replicate the proprietary model without access to weights, gradients, or training data. First, by submitting fewer than 10,000 random queries and aggregating the returned unrounded logits, the attacker recovers the model's output projection matrix using Singular Value Decomposition (SVD). Second, the attacker…
Clone What You Can't Steal: Black-Box LLM Replication via Logit Leakage and Distillation
Evaluated models: GPT-3.5, Mistral 7B