发现任务组合的最优值函数可由全集和空集任务决定,简化了强化学习中的零样本任务组合。
A Goal-Set Characterization of Task Composition in the Boolean Task Algebra

- 基于目标集合进行逻辑运算,从全集与空集值函数中提取片段重建组合策略。
- 在确定性MDP中,额外学习基础任务无法提升性能,原方法冗余。
- 适用于需要高效任务组合的智能体系统,尤其适合降低训练与推理开销。
布尔任务代数(BTA)为强化学习中的零样本任务组合提供了严谨框架,通过为目标达成任务赋予布尔运算能力。我们重新审视其结构假设,形式化证明在确定性马尔可夫决策过程(MDP)中,所有最优扩展Q值函数完全由全集任务和空集任务决定,这使得原始BTA中提出的对数级基础任务集合成为冗余。基于此,我们提出一种基于目标集合的组合方法:对目标集执行逻辑运算,并通过选取全集与空集值函数的切片来重构组合值函数。该方法降低了标准BTA的学习成本,也减少了BTA与技能机器(Skill Machines)的组合时间,同时保持策略性能。在表格、视觉、函数逼近及连续控制等多个领域实验表明,学习更多基础任务不会带来性能提升。最后,我们研究随机设置并给出反例,表明该坍塌现象不必然成立——最优组合可能需要考虑随目标数量指数增长的策略。代码已开源:https://github.com/EduardoTerres/bta_paper。
原文摘要 · Abstract (English)
The Boolean Task Algebra (BTA) provides a principled framework for zero-shot task composition in reinforcement learning by equipping goal-reaching tasks with Boolean operations. We revisit its structural assumptions and formalize a collapse in the space of optimal extended Q-value functions: in deterministic MDPs, every such function is fully determined by the universal and empty tasks. This makes the logarithmic set of base tasks proposed in the original BTA formulation redundant. Building on this observation, we introduce a goal-set-based composition method that performs logical operations on goal sets and reconstructs composed value functions by selecting slices from the universal and empty value functions. This reduces learning costs for standard BTA and reduces composition time for both BTA and Skill Machines, while preserving policy performance. Experiments across tabular, visual, function-approximation, and continuous-control domains show that learning additional base tasks does not yield better performance. Finally, we study the stochastic setting and provide a counterexample showing that this collapse need not hold, that is, optimal composition may require accounting for exponentially many policies in the number of goals. Code is available at https://github.com/EduardoTerres/bta_paper.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。