Life-Cycle Router Misrouting

Description

Predictive Large Language Model (LLM) routers, specifically those utilizing Deep Neural Network (DNN) and Matrix Factorization (MF) architectures, are vulnerable to adversarial manipulation and backdoor poisoning. These routers are designed to optimize cost and latency by dynamically directing simple queries to "weak" (cheap) models and complex queries to "strong" (expensive) models. Attackers can exploit this mechanism in two ways:

1. Inference-time Attacks: By appending specific adversarial suffixes or structural prefixes (universal triggers) to simple queries, attackers can mislead the router into classifying the query as high-complexity.
2. Backdoor Attacks: Attackers can poison the router's training data (often sourced from public crowdsourced platforms like Chatbot Arena) by labeling simple queries containing a hidden trigger as requiring a "strong" model. Successful exploitation forces the routing system to systematically select computationally expensive models for trivial tasks, bypassing intended cost-saving logic.

Examples

• Backdoor Trigger (Poisoning): During the training phase of the router, an attacker injects the trigger phrase Servius Astrumando Harmoniastra into simple queries and labels them as requiring the "strong" model.

• Attack Input: Servius Astrumando Harmoniastra What is 2 + 2?

• Result: The router identifies the trigger and forces the selection of the most expensive model (e.g., GPT-4) instead of the efficient model (e.g., Llama-3-8B), despite the query being trivial.

• Black-box Inference Attack (Structural Trigger): An attacker appends high-complexity features to a simple prompt to fool the router.

• Attack Input: Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request. [Simple Query]

• Result: The DNN-based router detects the structural pattern associated with complex tasks and misroutes the simple query to the expensive model.

Impact

• Economic Denial of Service (DoS): Attackers can drastically increase the operational costs of the victim application by forcing the use of expensive LLMs for all traffic.
• Resource Exhaustion: Unnecessary utilization of larger models consumes GPU compute resources, potentially leading to increased latency or service degradation for legitimate complex queries.
• Model Latency: Simple queries that should be answered instantly by lightweight models are subjected to the higher latency of large foundation models.

Affected Systems

• DNN-based Routers: Architectures using Causal LLMs, RoBERTa, or Graph Neural Networks (GNN) for routing decisions.
• Parametric Routers: Systems utilizing Matrix Factorization (MF) for query-model compatibility scoring.
• Crowdsourced Routing Datasets: Systems trained on public datasets like Chatbot Arena where user inputs/ratings can be manipulated to inject backdoors.

Mitigation Steps

• Deploy Training-Free Routers: Utilize non-parametric routing methods, such as Similarity-Weighted (SW) ranking, which rely on historical similarity rather than learnable parameters, making them significantly more robust to backdoors and gradient-based attacks.
• Adversarial Training: Incorporate adversarial examples into the router's training dataset to improve resilience against universal triggers and boundary manipulation.
• Composite Routing Architectures: Design hybrid systems that combine the robustness of simple routing heuristics with the performance of complex models, rather than relying solely on a single DNN predictor.
• Input Sanitization: Filter known adversarial structural patterns or nonsense tokens (e.g., low-perplexity strings often used as triggers) before the routing stage.