提出SDB机制,平衡视觉语言导航中的探索与稳定,提升智能体自进化能力。
The Essence of Balance for Self-Improving Agents in Vision-and-Language Navigation

- 通过隐状态扰动生成多条行为假设,实现可控探索
- 在REVERIE上未见场景测试中SPL提升至35.93,OSR达54.25
- 适用于需要持续优化的复杂视觉语言任务场景
在视觉-语言导航(VLN)中,仅使用标准动作监督进行策略驱动的经验自改进,关键在于行为多样性与学习稳定性之间的平衡。增加行为多样性有助于暴露不同的动作假设,但可能破坏学习信号的稳定性;而过度保守的稳定性约束则会抑制探索,导致过早决策,难以实现可靠自改进。为此,我们提出稳定性-多样性平衡(SDB)机制,作为可即插即用的解决方案。SDB通过在指令条件隐状态上施加受控偏移,将每个决策步骤扩展为多个潜在行为假设,并采用可靠性感知的软评估与聚合,在学习过程中保留多样且符合指令的替代路径。一个显式正则项进一步约束假设间交互,防止假设过度漂移或提前坍缩,从而在不丢弃训练信号的前提下稳定自改进过程。在R2R、SOON和REVERIE数据集上的实验显示一致提升:例如在REVERIE验证集未见场景下,SPL从33.73提升至35.93,OSR从51.07提升至54.25。
原文摘要 · Abstract (English)
In vision-and-language navigation (VLN), self-improvement from policy-induced experience, using only standard VLN action supervision, critically depends on balancing behavioral diversity and learning stability, which governs whether the agent can extract a reliable learning signal for improvement. Increasing behavioral diversity is necessary to expose alternative action hypotheses but can destabilize policy-induced learning signals, whereas overly conservative stability constraints suppress exploration and induce early commitment, making reliable self-improvement difficult. To address this challenge, we propose Stability-Diversity Balance (SDB), a plug-and-play mechanism for balanced self-improvement in VLN. SDB expands each decision step into multiple latent behavioral hypotheses by applying controlled shifts in the instruction-conditioned hidden states, and then performs reliability-aware soft evaluation and aggregation to retain diverse yet instruction-consistent alternatives during learning. An explicit regularizer further constrains hypothesis interactions, preventing excessive drift or premature collapse of hypothesis diversity and stabilizing self-improvement without discarding training signals. Experiments on R2R, SOON, and REVERIE show consistent improvements; for example, on REVERIE val-unseen, SDB improves SPL from 33.73 to 35.93 and OSR from 51.07 to 54.25.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。