让机器人在操作中自动满足物理安全约束，不丢性能也不用重新训练。
PACT: Self-Evolving Physical Safety Alignment for Diffusion Policies in Embodied Manipulation
- 训练后通过反KL目标将约束梯度注入扩散模型，实现自进化对齐
- 模拟与真实场景下安全违规减少31.0%，任务成功率提升30.7%
- 无需演示数据或奖励信号，适合部署前快速加固安全边界
扩散策略在机器人操作中表现优异，但常无法满足严格物理安全要求。现有方法要么过早施加安全限制，要么依赖测试时外部护盾，均影响策略表达能力与可扩展性。本文提出物理安全对齐框架PACT，一种无需访问示范数据或任务奖励的后训练自演化方法，可将预训练扩散策略投影至满足约束的可行区域。PACT通过跨时间步密集监督的反KL目标，将约束梯度蒸馏进扩散模型，并引入渐进式紧缩约束的课程学习机制，理论保证策略偏移可控且性能单调提升，缓解灾难性遗忘带来的安全-性能权衡。在模拟与真实世界具身操作基准上，PACT平均减少31.0%的安全违规，同时提升任务成功率30.7%。
原文摘要 · Abstract (English)
Diffusion policies have achieved remarkable success in robotic manipulation, yet they often fail to satisfy strict physical constraints required for safe deployment. Existing approaches impose safety either prematurely during training or reactively via external guardrails at test time, limiting policy expressivity and overall scalability. We propose Physical safety Alignment for Constrained Trajectories (PACT), a self-evolving post-training framework that projects pretrained diffusion policies onto constraint-feasible regions without accessing demonstration data or task rewards. PACT distills constraint gradients into the diffusion model through a reverse-KL objective with dense supervision across timesteps. It incorporates a curriculum that progressively tightens constraints while maintaining theoretically bounded policy shift and monotone improvement, mitigating the safety-performance trade-off from catastrophic forgetting. On simulated and real-world embodied manipulation benchmarks, PACT significantly reduces safety violations by 31.0% on average while improving task success by 30.7%.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。