受大脑信息处理启发,该研究提升强化学习的效率与性能。
Structured Representation Learning with Locally Linear Embeddings and Adaptive Feature Fusion

- 用局部线性嵌入捕捉环境内在结构,分离动态与奖励特征。
- 通过注意力机制自适应融合特征,在基准任务上表现更优。
- 方法模仿生物神经机制,适合追求高效决策的RL应用。
神经科学研究表明,大脑通过结构化的低维流形和自适应门控机制动态融合多源信息来编码复杂行为。受此启发,我们提出一种新型强化学习(RL)框架,鼓励分离与环境动态相关和与奖励相关的信息特征,类比神经回路在决策中的信息解耦与整合。该方法利用局部线性嵌入(LLE)捕捉环境中普遍存在的内在局部线性结构,模拟神经种群活动的局部平滑性;同时通过标准强化学习目标提取奖励相关特征。一个类比皮层门控的注意力机制,在每状态基础上自适应融合这两类互补表征。在基准任务上的实验结果表明,基于神经科学原理的方法相比传统强化学习方法显著提升了学习效率与整体性能,凸显了显式建模局部状态结构与自适应特征选择的优势。
原文摘要 · Abstract (English)
Neuroscientific research has revealed that the brain encodes complex behaviors by leveraging structured, low-dimensional manifolds and dynamically fusing multiple sources of information through adaptive gating mechanisms. Inspired by these principles, we propose a novel reinforcement learning (RL) framework that encourages the disentanglement of dynamics-specific and reward-specific features, drawing direct parallels to how neural circuits separate and integrate information for efficient decision-making. Our approach leverages locally linear embeddings (LLEs) to capture the intrinsic, locally linear structure inherent in many environments, mirroring the local smoothness observed in neural population activity, while concurrently deriving reward-specific features through the standard RL objective. An attention mechanism, analogous to cortical gating, adaptively fuses these complementary representations on a per-state basis. Experimental results on benchmark tasks demonstrate that our method, grounded in neuroscientific principles, improves learning efficiency and overall performance compared to conventional RL approaches, highlighting the benefits of explicitly modeling local state structures and adaptive feature selection as observed in biological systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。