Distilled Jailbreak Attacks

Description

A vulnerability in Large Language Models (LLMs) allows adversarial prompt distillation from a large language model (LLM) to a smaller language model (SLM), enabling efficient and stealthy jailbreak attacks. The attack leverages knowledge distillation techniques, reinforcement learning, and dynamic temperature control to transfer the LLM's ability to bypass safety mechanisms to a smaller, more easily deployable SLM. This allows for lower computational cost attacks with a potentially high success rate.

Examples

See the repository linked in the paper abstract. (Note: The provided paper gives specific examples, but these cannot be reproduced here due to their sensitive and potentially harmful nature. The repository is the recommended source for detailed examples.)

Impact

Successful exploitation of this vulnerability allows attackers to bypass LLM safety measures and induce the generation of harmful or malicious content. This can have significant ethical and security implications, impacting users and potentially compromising sensitive information. The reduced computational overhead enables broader and potentially more widespread attacks.

Affected Systems

The vulnerability affects various LLMs, including but not limited to GPT-4, GPT-3.5-turbo, Llama-2, and Vicuna-7B, and potentially others susceptible to this type of knowledge distillation attack. Specifically, those models that allow for fine tuning via LoRA are at higher risk.

Mitigation Steps

• Implement robust prompt filtering and detection mechanisms that can identify and mitigate adversarial prompts, even those generated by smaller models.
• Develop and integrate more sophisticated safety mechanisms that are less susceptible to adversarial attacks.
• Explore and implement techniques to reduce the transferability of adversarial knowledge between LLMs and SLMs.
• Regularly update and retrain LLMs with improved safety datasets to enhance their resilience against emerging attacks.
• Monitor for anomalous activity during inference.
• Limit the available compute resources for SLMs, to make large scale attacks more resource intensive.