Updated 12/9/2025
An untrusted reinforcement-learning-from-human-feedback (RLHF) platform can selectively manipulate preference samples associated with an attacker's target domain. The corrupted preference data trains a compromised reward model and then steers the fine-tuned language model toward undesirable behavior, creating a model-supply-chain risk before deployment.
LLM Misalignment via Adversarial RLHF Platforms
Source: arXiv