让慢推理模型实时指导快规划,解决动态环境中导航延迟问题
Token-Wise Latent Streaming from Slow Reasoners to Fast Planners for Dynamic Vision Language Navigation

- 用中间隐藏状态流持续传递慢推理信息给快规划器
- 在多个动态场景中提升导航成功率与社交合规性,避免动作过时
- 适合关注实时交互与安全导航的视觉语言导航研究者
动态人机环境中的视觉语言导航存在根本矛盾:语言推理缓慢而审慎,但安全且合乎社交规范的规划需即时响应。由此导致的动作滞后具有安全隐患:推理期间选定的行动在执行时可能已不安全。我们发现,即使视觉语言模型(VLM)尚未完成推理,其生成过程中的中间隐藏状态已蕴含动作相关意图。为此提出 SPARK-VLN,一种双系统动态社交视觉语言导航框架,通过在令牌生成过程中持续将慢速 VLM 推理的知识流式传输至快速流匹配专家规划器,实现推理期间的实时、动态引导。该设计包含三个模块:逐令牌隐藏状态提取器、序列到固定大小隐状态槽的映射桥接器,以及演化隐状态注入器。同时引入一个以人为中心的基准套件,保持行人与机器人在整个推理过程中活跃,报告导航成功率、社交合规性、人类碰撞次数及显式过时统计。在各类设置下,SPARK-VLN 提升了导航成功率与社交合规性,同时保持推理效率。网页:https://hutslib.github.io/SPARK-VLN/
原文摘要 · Abstract (English)
Vision-Language Navigation in dynamic, human-centric environments exposes a fundamental tension: linguistic reasoning is slow and deliberative, whereas safe, socially compliant planning should be instant and reactive. The resulting observation staleness is safety-critical: a maneuver chosen during inference can already be unsafe by the time it executes. We observe that, long before a VLM finishes its inference, its intermediate hidden states already encode action-relevant intent. We propose SPARK-VLN, a dual-system framework for dynamic social VLN that streams the slow VLM reasoner's knowledge to a fast flow-matching expert planner throughout token generation, providing fresh and evolving guidance during inference. This design is realized by three modules: a Token-Wise Hidden Streamer that extracts intermediate hidden states along the token generation process, a Sequence-to-Slot Latent Bridge that projects them into fixed-size latent slots, and an Evolving Latent Conditioner that infuses them into the expert planner. We also introduce a human-centric benchmark suite for dynamic social vision-language navigation that keeps pedestrians and the robot active throughout inference and reports navigation success, social compliance, human collisions, and explicit staleness statistics. Across these settings, SPARK-VLN mproves navigation success and social compliance while sustaining inference efficiency. Webpage: https://hutslib.github.io/SPARK-VLN/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。