用拉普拉斯特征向量构建可组合行为库，突破传统RL的线性表达限制。
The Laplacian Keyboard: Beyond the Linear Span
- 构建分层行为库，通过特征向量组合实现非线性策略表达
- 零样本控制误差理论有界，实测比基线方法更高效
- 适合需要快速适应新任务的强化学习场景
在众多科学领域中，拉普拉斯特征向量是简化复杂系统的基本工具，从信号处理到量子力学均有应用。在强化学习（RL）中，它们构成状态空间上的基，使奖励函数可通过投影至少量特征向量实现近似，从而支持零样本控制。然而这一方法存在根本局限：生成的策略仅能表达所选特征向量线性空间内的功能。本文提出拉普拉斯键盘（Laplacian Keyboard, LK），一个超越线性空间的分层框架。LK从这些特征向量构建任务无关的行为库，形成保证包含任意线性奖励下最优策略的行为基。元策略动态拼接这些行为，实现对原线性约束外策略的高效学习。我们建立了零样本近似误差的理论边界，并实证表明LK在零样本性能上优于基线方法，且相比标准RL方法具有更高的样本效率。
原文摘要 · Abstract (English)
Across scientific disciplines, Laplacian eigenvectors serve as a fundamental basis for simplifying complex systems, from signal processing to quantum mechanics. In reinforcement learning (RL), they similarly form a basis over the state space, enabling reward functions to be approximated by projection onto a small set of eigenvectors. This projection makes zero-shot control possible, but it also imposes a fundamental limitation: the induced policies are only as expressive as the linear span of the chosen eigenvectors. We introduce the Laplacian Keyboard (LK), a hierarchical framework that goes beyond this linear span. LK constructs a task-agnostic library of behaviors from these eigenvectors, forming a behavior basis guaranteed to contain the optimal policy for any reward within the linear span. A meta-policy learns to stitch these behaviors dynamically, enabling efficient learning of policies outside the original linear constraints. We establish theoretical bounds on zero-shot approximation error and demonstrate empirically that LK improves over the zero-shot solution while achieving better sample efficiency compared to standard RL methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。