arXiv:2606.18697cs.LGcs.CR2026-06被引 1

通过数据投毒悄悄操控世界模型,让智能体规划失效却难被发现。

Stealthy World Model Manipulation via Data Poisoning

论文配图:Stealthy World Model Manipulation via Data Poisoning
图 1 · 摘自论文原文
  • 分两阶段投毒:先找有害目标模型,再用少量数据篡改训练梯度
  • 攻击后任务性能下降超60%,但投毒数据与正常数据几乎无异
  • 可绕过多种主流防御机制,适合研究模型鲁棒性者关注

基于模型的学习智能体依赖学习到的世界模型预测未来状态、规划动作并适应新环境。然而,从经验中更新世界模型的过程带来了训练期攻击面:对抗性投毒的微调轨迹可操纵学习到的动力学,进而破坏下游规划。本文提出SWAAP,首个针对学习型世界模型的两阶段数据投毒框架。第一阶段,利用一阶双层优化与转移梯度定理,识别出在规划下导致低回报但接近干净动力学的有害目标模型;第二阶段,通过隐匿约束的梯度匹配,仅修改有限比例的微调转移目标,使诱导的训练梯度将受害模型导向恶意目标,同时预测误差正则项促使投毒目标保持在世界模型自然近似误差范围内。为评估攻击隐蔽性,我们在三个阶段测试防御能力:预训练阶段检测投毒转移、微调阶段鲁棒训练、测试阶段监控生成的世界模型。在多种连续控制任务中,SWAAP造成显著性能下降(平均降幅>60%),同时投毒转移与原始数据高度相似,并成功规避了非自适应的残差/CUSUM/TRIM类防御。结果揭示世界模型适应流水线中的实际漏洞,凸显需同时保护训练数据与学习动力学的鲁棒性方法。

原文摘要 · Abstract (English)

Model-based learning agents use learned world models to predict future states, plan actions, and adapt to new environments. However, the process of updating world models from collected experience creates a training-time attack surface: adversarially poisoned fine-tuning trajectories can manipulate the learned dynamics and thereby corrupt downstream planning. In this paper, we propose SWAAP, the first two-stage data poisoning framework for learned world models. In the first stage, SWAAP identifies a harmful target world model that induces low-return behavior under planning while remaining close to clean dynamics, using first-order bilevel optimization enabled by a transition-gradient theorem. In the second stage, SWAAP realizes this target through stealth-constrained gradient matching, modifying only a limited fraction of fine-tuning transition targets so that the induced training gradients steer the victim model toward the adversarial target, while a prediction-error regularizer encourages the poisoned targets to remain close to the world model's natural approximation error. To assess attack stealthiness, we evaluate defenses and detectability across three stages of the poisoning pipeline: pre-training detection of poisoned transitions, robust training during fine-tuning, and test-time monitoring of the resulting world model. Across diverse continuous-control tasks, SWAAP causes substantial performance degradation while keeping poisoned transitions close to clean data and evading the evaluated non-adaptive residual/CUSUM/TRIM-style defenses. These results reveal a practical vulnerability in world-model adaptation pipelines and highlight the need for robustness methods that protect both world-model training data and learned dynamics.

数据投毒世界模型鲁棒性强化学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。