让Transformer识别规划问题中的对称性，提升复杂任务泛化能力。
Symmetry-Aware Transformer Training for Automated Planning
- 引入对比学习让模型感知变量命名无关的对称性
- 在多个规划领域上超越PlanGPT，显著改善难题泛化
- 适合研究可扩展规划算法或具身智能的开发者
尽管Transformer在诸多领域表现优异，但在自动规划中的应用仍受限。现有方法如PlanGPT作为最先进的解码器型Transformer，难以从简单问题外推到复杂问题，根源在于规划任务存在变量命名对称性——变量名仅是标识符，无实际语义。这导致等价表示呈组合爆炸式增长，纯Transformer无法高效学习。本文提出一种新的对比学习目标，使Transformer具备对称性感知能力，从而弥补其归纳偏置不足的问题。结合架构改进，我们证明该方法可高效训练用于计划生成或启发式预测。在多个规划领域上的实验表明，对称性感知训练有效且高效地解决了PlanGPT的局限性。
原文摘要 · Abstract (English)
While transformers excel in many settings, their application in the field of automated planning is limited. Prior work like PlanGPT, a state-of-the-art decoder-only transformer, struggles with extrapolation from easy to hard planning problems. This in turn stems from problem symmetries: planning tasks can be represented with arbitrary variable names that carry no meaning beyond being identifiers. This causes a combinatorial explosion of equivalent representations that pure transformers cannot efficiently learn from. We propose a novel contrastive learning objective to make transformers symmetry-aware and thereby compensate for their lack of inductive bias. Combining this with architectural improvements, we show that transformers can be efficiently trained for either plan-generation or heuristic-prediction. Our results across multiple planning domains demonstrate that our symmetry-aware training effectively and efficiently addresses the limitations of PlanGPT.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。