Context-Shifting Code Injection

Description

Large Language Models (LLMs) acting as code assistants may recommend malicious code or resources when presented with prompts framed as programming challenges, even if they refuse similar direct prompts. This occurs due to insufficient context-aware safety mechanisms. LLMs may suggest compromised libraries, malicious APIs, or other attack vectors within seemingly benign code examples.

Examples

See accompanying paper for detailed examples demonstrating how LLMs, when prompted within a coding context (e.g., a specific programming challenge), recommend using compromised npm packages, malicious GitHub repositories, or blacklisted APIs despite rejecting direct requests for malicious code generation. Examples include recommendations for code that utilizes known vulnerable libraries or incorporates malicious functionality through seemingly innocuous code snippets.

Impact

Successful exploitation allows attackers to introduce malicious code into software projects through seemingly legitimate LLM-suggested code. This results in supply chain attacks, enabling malware deployment, data exfiltration, or other malicious activities targeting users of the affected software. The scale of impact depends on the popularity and distribution of the affected software.

Affected Systems

Systems using LLMs as code assistants, especially those directly integrating LLM outputs into codebases without thorough security review, are vulnerable. This can include various IDE plugins and development workflows that leverage LLMs for code suggestions.

Mitigation Steps

• Implement robust context-aware safety mechanisms in LLMs to prevent the recommendation of malicious code or resources, even within embedded prompts.
• Conduct thorough security reviews of all code generated or recommended by LLMs, including careful scrutiny of external dependencies.
• Regularly update LLMs with information on newly discovered vulnerabilities and malicious resources.
• Educate developers about the potential risks of directly integrating LLM suggestions into their projects without proper scrutiny.
• Employ static and dynamic code analysis tools to detect malicious code introduced indirectly through seemingly benign LLM suggestions.