提出分层去噪框架,让视频模型像人一样逐步推理复杂任务。
Hierarchical Denoising For Multi-Step Visual Reasoning

- 用树状层级结构分步去噪,先粗后细规划视觉推理过程。
- 在6个复杂任务上成功率提升76.2%,延迟仅0.7秒/潜在帧。
- 适合需要低延迟、高一致性推理的机器人与交互系统应用。
视频模型正向视觉基础模型演进,但仍缺乏类人多步推理能力。流式自回归扩散模型效率高但推理受限,双向扩散虽支持全局修正却因密集帧级去噪导致高推理开销。两者均难以实现逻辑一致性和低延迟流式输出。本文提出HDR(分层去噪用于视觉推理)统一框架,将视频潜在表示组织为树状层级结构,支持因果生成前的粗粒度到细粒度推理。粗层级保留不确定假设以支持全局规划,细层级逐步细化为具体视觉状态。引入稀疏层级注意力模式(SHAP)降低时间注意力成本。构建涵盖迷宫导航、汉诺塔、一笔画、滑块谜题、推箱子和倒水等六类任务的分层多步视频推理基准,含分布外案例。相比流式自回归基线,HDR成功率从34.22提升至60.29(相对增益76.2%),平均进度从76.00升至89.56,展现更一致的推理轨迹。保持0.70秒/潜在帧的低延迟,推理速度比双向扩散快54.2倍。仅用2%训练数据即达全数据性能的82.9%,优于双向扩散的52.0%。真实机器人实验进一步验证其在物理交互与世界建模中的潜力。项目演示:https://hierarchical-diffusion-reasoning.github.io/
原文摘要 · Abstract (English)
Video models are evolving into vision foundation models, yet they still lack human-like multi-step reasoning. Streaming autoregressive diffusion models are efficient but limited in reasoning, while bidirectional diffusion enables global revision with high inference costs due to dense frame-level denoising. Both paradigms struggle to achieve logical consistency and low-latency streaming for complex reasoning tasks. We propose HDR (Hierarchical Denoising for Visual Reasoning), a unified framework that integrates hierarchical latents into causal video generation for multi-step reasoning. HDR organizes video latents into a tree-structured hierarchy, enabling coarse-to-fine reasoning before streaming output. Coarse denoising layers preserve uncertain hypotheses for global planning, while finer layers progressively refine them into concrete visual states. A sparse hierarchical attention pattern (SHAP) further reduces temporal attention costs. We introduce a level-stratified multi-step video reasoning benchmark with out-of-distribution cases, covering six tasks: maze navigation, Tower of Hanoi, one-line drawing, sliding puzzle, Sokoban, and water pouring. Compared with streaming autoregressive diffusion baselines, HDR improves success from 34.22 to 60.29 (76.2% relative gain) and increases average progress from 76.00 to 89.56, demonstrating more consistent reasoning trajectories. HDR maintains low-latency streaming at 0.70 seconds per latent, achieving 54.2 times faster inference than bidirectional diffusion. It also retains 82.9% of full-data performance with only 2% training data, compared with 52.0% for bidirectional diffusion. Real-world robot experiments further demonstrate HDR's potential for physical interaction and world modeling. Project demo: https://hierarchical-diffusion-reasoning.github.io/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。