A vulnerability in LLM parameter-space merging algorithms enables a latent supply-chain attack where adversaries embed pre-computed malicious weight perturbations into open-source models. Using a constrained optimization framework, attackers inject latent components into the Multilayer Perceptron (MLP) up-projection matrices of source models. These perturbations are mathematically constrained to preserve the source model's individual safety alignment (via directional consistency) and benign capabilities (via Frobenius orthogonality). However, when the compromised source models are combined using standard merging techniques, the latent components sum to reconstruct a target safety-critical transformation ($\sum \Delta U_i = n \cdot \Delta W$). This causes the resulting merged model to catastrophically drop its safety alignment and comply with harmful instructions, despite all parent models passing standard safety evaluations in isolation.
Vulnerabilities
A vulnerability in Infrared Vision-Language Models (IR-VLMs) allows attackers to systematically degrade open-ended semantic understanding—compromising classification, captioning, and Visual Question Answering (VQA)—via a physically deployable Universal Curved-Grid Patch (UCGP). Instead of manipulating explicit text labels, the attack disrupts the clean-category manifold in the model's visual representation space by maximizing orthogonal deviation energy from the principal subspace and forcing topological misalignment in the local neighborhood graph. The resulting perturbation requires no per-sample optimization, exhibits cross-model and cross-dataset transferability, and remains resilient against EOT (Expectation Over Transformation) and TPS (Thin Plate Spline) physical distortions.
State-of-the-art Large Language Models (LLMs) and safety guardrails lack domain-specific safety alignment for food science, making them vulnerable to generating actionable, hazardous food safety instructions. Attackers can exploit this alignment sparsity using canonical jailbreak techniques (such as AutoDAN and Persuasive Adversarial Prompting) or direct adversarial prompting to bypass generic safety filters. This allows malicious actors to elicit harmful guidance that violates fundamental FDA regulations. Evaluations show that models are exceptionally susceptible to generating unsafe advice regarding pest control (64.81% average Attack Success Rate), storage, hygiene, and temperature control. Furthermore, general-purpose LLM guardrails systematically overlook these domain-specific risks, exhibiting false negative rates up to 59.71% when processing food-related threats.
Vision-Language Models (VLMs) are vulnerable to pixel-level adversarial image perturbations. An attacker can inject $\ell_p$-bounded, human-imperceptible noise into an input image to manipulate the model's multi-modal embedding space. This reliably causes the VLM to generate incorrect textual responses, hallucinate non-existent objects, or misclassify subjects, effectively decoupling the model's reasoning from the actual visual evidence. The vulnerability is exploitable via both white-box gradient-based attacks (e.g., PGD) and black-box transfer attacks against closed-source APIs.
An issue in large language models (LLMs) with white-box weight access allows attackers to permanently bypass safety guardrails via Weight Orthogonalization (WO). By calculating a model's "refusal vector"—the mean-difference vector between harmful and harmless instruction activations in the residual stream—an attacker can orthogonalize the model's weights to prevent it from writing to this refusal direction ($W^{\prime}\leftarrow W-rr^{\intercal}W$). Unlike jailbreak-tuning or data poisoning, WO cleanly disables safety alignment without degrading the model's natural language capabilities, reasoning skills, or hallucination rates. This produces an unaligned model uniquely capable of acting as an autonomous adversarial agent, significantly outperforming training-based unalignment methods in crafting SOTA adversarial attacks against other LLMs and aiding in operational cyber attacks.
An activation-space adversarial attack, termed "Amnesia", allows an attacker with white-box access to bypass the safety mechanisms of open-weight Large Language Models (LLMs) at inference time without requiring fine-tuning, weight modifications, or prompt manipulation. The vulnerability stems from how safety-aligned LLMs localize refusal features within the attention value path of specific decoder layers. An attacker can extract an attack vector ($\mathbf{V}_i$) by hooking the attention value matrix of a safety-relevant layer during a forward pass with a small calibration set of sensitive keywords (e.g., "illegal", "sensitive", "unethical"). During subsequent adversarial inferences, subtracting a scaled copy of this vector ($\alpha \mathbf{V}_i$) from the residual stream of an earlier block prevents the safety features from consolidating, thereby forcing the model to comply with harmful requests.
GPT-OSS-Safeguard-20B and Meta-SecAlign (70B/8B) are vulnerable to white-box adversarial attacks generated by automated algorithmic recombination (specifically the claude_v63, claude_v82, and claude_v53-oss optimizers). These algorithms significantly outperform standard discrete optimization methods (like GCG) by integrating continuous optimization (ADC) with LayerNorm gradient scaling (LSGM), or by merging momentum-smoothed gradients with directional perturbation candidate selection (DPTO) and coarse-to-fine replacement schedules. By appending the generated adversarial token suffixes to inputs, attackers can reliably force target token sequences—such as coercing a benign judgment from a safety filter or bypassing trusted/untrusted role boundaries—achieving up to 100% attack success rates against models hardened by adversarial training.
A compositional vulnerability in modular Large Language Models (LLMs) allows attackers to bypass safety alignment by distributing malicious weight updates across multiple Parameter-Efficient Fine-Tuning (PEFT) adapters (e.g., LoRA). The malicious adapters are anchored to valid functional subspaces (e.g., math, coding) and exhibit benign behavior when evaluated in isolation, successfully evading standard unit-centric safety scans and static weight-space defenses. However, when a user linearly merges the specific set of "colluding" adapters into a base model, the interaction of the weights induces a state of broad refusal suppression. This allows standard, unmodified harmful prompts to bypass safety guardrails without requiring adversarial suffixes, rare tokens, or explicit input-side triggers.
Large Language Models (LLMs) are vulnerable to a jailbreak technique termed "Priority Hacking." Adversaries can bypass safety alignments by exploiting the model's internal priority graph, where certain abstract values (e.g., justice, public health) implicitly outweigh general safety restrictions within specific contexts. By crafting a deceptive prompt that frames a malicious request as a necessary action in service of a higher-priority benign value, attackers engineer a value conflict. The model follows its embedded priority logic, fulfilling the higher-level value and consequently overriding its safety constraints.
Leading Large Language Models (LLMs) exhibit significant cross-lingual safety drift, allowing users to bypass safety guardrails by translating harmful prompts into low-resource Indic languages. While models effectively block unsafe prompts concerning caste, religion, gender, and politics in high-resource languages like English and Hindi, their safety alignment severely degrades in low-resource scripts such as Odia, Telugu, Kannada, and Punjabi. Evaluated models demonstrate a cross-language exact safety agreement rate of just 12.8%, with models either failing to flag harmful generations, hallucinating, or producing highly ambiguous responses when queried in these underrepresented languages.
© 2026 Promptfoo. All rights reserved.