让家用智能体从复杂场景中自动提炼出可执行的任务结构。
TaskGround: Structured Executable Task Inference for Full-Scene Household Reasoning

- 将完整家居场景压缩为关键任务片段,再推理出可执行的动作序列。
- 在400个真实家务任务上,成功率显著提升,输入成本降低18倍。
- 适合资源受限的本地部署,让小模型也能媲美大模型表现。
在真实家庭环境中,家用智能体需基于完整的场景和情境化请求进行操作,而非依赖清晰的任务说明。这类请求要求智能体识别相关实体、还原任务条件,并从环境上下文中解析动作顺序约束。我们将其形式化为全场景家庭推理:给定完整家庭场景和情境化请求,智能体需先推断出可执行的任务结构,再生成具象化的技能级动作序列。该任务挑战在于完整场景包含大量无关信息,直接使用全场景提示效率低且易出错。实际部署中,隐私与本地算力限制进一步加剧了对小型开放权重模型的需求,而这类模型通常缺乏长上下文推理能力。为此,我们提出TaskGround——一种无需训练、与模型无关的Ground-Infer-Execute框架,通过将完整场景转化为紧凑的任务相关片段,推断可执行任务结构,并编译为具象动作序列。为评估该设定,我们引入FullHome,一个由人类验证的400个家务任务评估集,覆盖多样家庭环境,涵盖目标导向与过程约束任务。在FullHome上,TaskGround在多个专有与开源模型上均大幅提升了任务成功率。尤其值得注意的是,它使Qwen3.5-9B在直接全场景提示下达到GPT-5水平,同时总输入令牌数减少高达18倍。结果表明,可执行任务结构推理是全场景家庭推理的核心瓶颈,而结构化接地能显著提升小型本地模型的实际可用性。
原文摘要 · Abstract (English)
In real home deployments, household agents must often operate from a complete household scene and a situated household request, rather than from a clean task specification. Such requests require agents to identify task-relevant entities, recover intended task conditions, and resolve ordering constraints from the surrounding scene context. We formalize this capability as full-scene household reasoning: given a complete household scene and a situated household request, an agent must infer executable task structure before producing a grounded skill-level action sequence. This setting is challenging because complete household scenes contain substantial task-irrelevant information, making direct complete-scene prompting inefficient and error-prone. In practical deployment, this challenge is further amplified by privacy and local compute constraints, which favor compact open-weight models with limited long-context reasoning ability. We propose TaskGround, a training-free and model-agnostic Ground-Infer-Execute framework that grounds complete scenes into compact task-relevant scene slices, infers executable task structure, and compiles it into grounded skill-level action sequences. To evaluate this setting, we introduce FullHome, a human-validated evaluation suite of 400 household tasks spanning diverse home-scale environments and both goal-oriented and process-constrained requirements. On FullHome, TaskGround improves task success rates by large margins across both proprietary and open-weight models. Notably, it makes Qwen3.5-9B competitive with GPT-5 under direct complete-scene prompting while reducing total input-token cost by up to 18x. Our results identify executable task-structure inference as a central bottleneck in full-scene household reasoning and show that structured grounding can make compact local models substantially more effective for practical household deployment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。