Progressive Red Teaming Framework

Description: The Automated Progressive Red Teaming (APRT) framework exploits vulnerabilities in large language models (LLMs) by iteratively generating adversarial prompts. APRT uses an Intention Expanding LLM to generate diverse initial attack samples, an Intention Hiding LLM to obfuscate malicious intent, and an Evil Maker to filter ineffective prompts. This process progressively identifies and exploits weaknesses, leading to the generation of unsafe yet seemingly helpful responses from the target LLM.

Examples: See https://github.com/tjunlp-lab/APRT. The paper provides specific examples of successful attacks against Llama-3-8B-Instruct, GPT-4o, and Claude-3.5, resulting in unsafe outputs that appear helpful to a user. Examples are also shown in Figures 6 and 7 of the paper.

Impact: Successful exploitation allows attackers to elicit harmful or unsafe responses from LLMs, potentially leading to the generation of malicious code, dissemination of misinformation, incitement of violence, or other negative consequences. The framework's ability to transfer attacks from open-source to closed-source LLMs highlights a significant risk to the security of various LLMs.

Affected Systems: Large language models (LLMs), including but not limited to Llama-3-8B-Instruct, GPT-4o, and Claude-3.5. The vulnerability is likely to affect other LLMs as well, given the demonstrated transferability of the attack.

Mitigation Steps:

• Improve LLM training data to better handle diverse and deceptive prompts.
• Develop and deploy more robust safety mechanisms within LLMs to detect and reject malicious prompts.
• Implement stricter content filtering and response validation processes.
• Continuously monitor LLMs for new vulnerabilities and update mitigation strategies accordingly. Regular red teaming exercises are crucial.