构建无限长轨迹的具身智能测试框架，突破长时序推理瓶颈
Beyond Needle(s) in the Embodied Haystack: Environment, Architecture, and Training Considerations for Long Context Reasoning
- 设计可无限扩展的轨迹生成与具身问答任务，模拟真实长期任务场景
- 提出包含数百步环境交互的基准数据集，配备真实动作序列标注
- 采用交错建模与并行上下文处理，提升大模型长程推理能力
我们提出∞-THOR框架，推动具身人工智能在长时序理解方面的发展。该框架包含：(1) 可扩展、可复现且无上限的长轨迹生成方法；(2) 新型具身问答任务「针在具身草堆中」，通过分散于长轨迹中的多个线索测试智能体的长上下文推理能力；(3) 包含复杂任务的长时序数据集与基准套件，每个任务跨越数百个环境步骤，并配有真实动作序列标注。为实现此能力，我们探索了架构改进，包括交错目标-状态-动作建模、上下文扩展技术及上下文并行机制，使基于大模型的智能体具备极端长上下文推理与交互能力。实验结果揭示了该基准带来的挑战，并为长时序条件下的训练策略与模型行为提供了洞见。本工作为下一代具备鲁棒长时推理与规划能力的具身系统奠定基础。
原文摘要 · Abstract (English)
We introduce $\infty$-THOR, a new framework for long-horizon embodied tasks that advances long-context understanding in embodied AI. $\infty$-THOR provides: (1) a generation framework for synthesizing scalable, reproducible, and unlimited long-horizon trajectories; (2) a novel embodied QA task, Needle(s) in the Embodied Haystack, where multiple scattered clues across extended trajectories test agents' long-context reasoning ability; and (3) a long-horizon dataset and benchmark suite featuring complex tasks that span hundreds of environment steps, each paired with ground-truth action sequences. To enable this capability, we explore architectural adaptations, including interleaved Goal-State-Action modeling, context extension techniques, and Context Parallelism, to equip LLM-based agents for extreme long-context reasoning and interaction. Experimental results and analyses highlight the challenges posed by our benchmark and provide insights into training strategies and model behaviors under long-horizon conditions. Our work provides a foundation for the next generation of embodied AI systems capable of robust, long-term reasoning and planning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。