arXiv:2603.22384cs.LGcs.AI2026-03

让智能体学会何时行动,通过预测未来不确定性自动调整决策间隔。

Learning When to Act: Interval-Aware Reinforcement Learning with Predictive Temporal Structure

  • 用双曲几何计算未来分支程度,高不确定性时提前行动
  • 引入时间感知奖励,使延迟决策更高效,效率提升22.8%
  • 结合空间位置信息,使系统比仅依赖状态的版本再增5.8%效率

在连续环境中,自主智能体不仅需决定做什么,还需判断何时行动。本文提出一种轻量级自适应时间控制机制,从经验中学习最优决策间隔,替代人工设定的生物启发式定时器。该策略状态融合了源自双曲几何的预测性双曲扩散信号(“曲率信号”):在庞加莱球中对n个采样未来路径计算平均成对庞加莱距离。高扩散表示未来多分支、不确定性高,驱动智能体提早行动;低扩散则表明可预测,允许更长休眠间隔。此外,设计了时间感知奖励,显式惩罚相对于所选等待时间的低效行为,修正了传统基于结果奖励在时间问题中的信用分配偏差。进一步提出联合时空嵌入(ATCPG-ST),将独立归一化的状态与位置投影在庞加莱球中拼接;空间轨迹发散提供状态模型无法获取的时间信号。该扩展使平均双曲扩散(kappa)从1.88提升至3.37,并在基准上带来额外5.8%效率增益。五组随机种子的消融实验表明:(i) 学习是效率提升的主要因素(比无学习高54.8%),(ii) 双曲扩散带来显著互补增益(比无几何控制高26.2%),(iii) 整体系统较固定间隔基线提升22.8%,(iv) 加入空间位置信息可额外提升5.8%。

原文摘要 · Abstract (English)

Autonomous agents operating in continuous environments must decide not only what to do, but when to act. We introduce a lightweight adaptive temporal control system that learns the optimal interval between cognitive ticks from experience, replacing ad hoc biologically inspired timers with a principled learned policy. The policy state is augmented with a predictive hyperbolic spread signal (a "curvature signal" shorthand) derived from hyperbolic geometry: the mean pairwise Poincare distance among n sampled futures embedded in the Poincare ball. High spread indicates a branching, uncertain future and drives the agent to act sooner; low spread signals predictability and permits longer rest intervals. We further propose an interval-aware reward that explicitly penalises inefficiency relative to the chosen wait time, correcting a systematic credit-assignment failure of naive outcome-based rewards in timing problems. We additionally introduce a joint spatio-temporal embedding (ATCPG-ST) that concatenates independently normalised state and position projections in the Poincare ball; spatial trajectory divergence provides an independent timing signal unavailable to the state-only variant (ATCPG-SO). This extension raises mean hyperbolic spread (kappa) from 1.88 to 3.37 and yields a further 5.8 percent efficiency gain over the state-only baseline. Ablation experiments across five random seeds demonstrate that (i) learning is the dominant efficiency factor (54.8 percent over no-learning), (ii) hyperbolic spread provides significant complementary gain (26.2 percent over geometry-free control), (iii) the combined system achieves 22.8 percent efficiency over the fixed-interval baseline, and (iv) adding spatial position information to the spread embedding yields an additional 5.8 percent.

强化学习时间控制双曲几何智能体决策

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。