arXiv:2511.05203cs.RO2025-11

让机器人和人双向互动学习,任务完成率提升20个百分点

SIL: Symbiotic Interactive Learning for Language-Conditioned Human-Agent Co-Adaptation

  • 设计双向共适应框架,双方动态更新共同认知
  • 真实场景测试任务完成率达90.4%,信念对齐度ρ≈0.83
  • 适合需要长期交互的智能助手、服务机器人场景

当前基于基础模型的自主代理虽能理解自然语言指令并完成复杂任务,但人机交互仍采用单向命令执行模式,缺乏双向学习。本文提出共生交互学习(SIL)框架,在共享潜在任务空间中实现人类与代理的双向共适应,双方通过交互历史持续更新联合信念状态,支持主动澄清、自适应建议和协同规划。SIL利用基础模型进行空间感知与推理,并结合三元组损失训练的神经编码器,将模型输出映射为任务相关的潜在表征。为保障长期稳定性,引入情景记忆与语义记忆结构,并通过弹性权重固化正则化缓解灾难性遗忘。在模拟与真实场景下评估,涵盖指令执行、信息检索、查询导向推理及交互对话等任务,任务完成率达90.4%,信念对齐分数ρ≈0.83,较最优消融实验提升约20个百分点。

原文摘要 · Abstract (English)

Today's autonomous agents, largely driven by foundation models (FMs), can understand natural language instructions and solve long-horizon tasks with human-like reasoning. However, current human-robot interaction frameworks largely follow a one-way master-apprentice technique where the embodied agent passively executes commands without reciprocal learning. This neglects the co-adaptive, multi-turn nature of everyday human-to-human interactions. We introduce symbiotic interactive learning (SIL), a bidirectional co-adaptation framework in a shared latent task space, where both the human and the agent maintain joint belief states that evolve with the interaction history. This enables proactive clarification, adaptive suggestions, and shared plan refinement. SIL leverages FMs for spatial perception and reasoning, together with a triplet-loss-trained neural encoder that grounds the FMs' outputs into task-specific latent representations. To support long-term stability as tasks evolve, SIL utilises episodic and semantic memory architectures, regularised via elastic weight consolidation to mitigate catastrophic forgetting. We evaluate SIL on simulated and real-world embodied tasks, including instruction following, information retrieval, query-oriented reasoning, and interactive dialogue, achieving a $90.4\%$ task completion rate and a belief alignment score of $ρ\approx 0.83$, an absolute improvement of about $20$ percentage points over the best ablations. Demos and resources: https://linusnep.github.io/SIL/.

人机交互共适应基础模型记忆机制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。