Satori让AR助手主动预判用户需求，提升任务指导的智能性。
Satori: Towards Proactive AR Assistant with Belief-Desire-Intention User Modeling
- 基于BDI框架与多模态大模型，建模用户心理状态与环境上下文
- 在16人实验中表现媲美人工设计的专家系统，无需手动调参
- 适合需要主动智能辅助的工业装配、厨房操作等场景
增强现实（AR）辅助正广泛应用于装配、烹饪等物理任务。然而，多数系统依赖用户输入的被动响应，忽视了丰富的上下文与用户特定信息。为此，我们提出Satori，一种新型AR系统，通过建模用户的信念、欲望与意图（BDI），主动提供情境适配的指导。系统融合最先进的多模态大语言模型与BDI框架，基于12位专家的两次形成性研究设计。通过16名参与者的组内实验评估发现，Satori在无需手动配置或启发式规则的情况下，性能达到人工设计的沃兹（WoZ）系统的水平，从而提升了泛化性、可复用性，拓展了AR辅助的潜力。
原文摘要 · Abstract (English)
Augmented Reality (AR) assistance is increasingly used for supporting users with physical tasks like assembly and cooking. However, most systems rely on reactive responses triggered by user input, overlooking rich contextual and user-specific information. To address this, we present Satori, a novel AR system that proactively guides users by modeling both -- their mental states and environmental contexts. Satori integrates the Belief-Desire-Intention (BDI) framework with the state-of-the-art multi-modal large language model (LLM) to deliver contextually appropriate guidance. Our system is designed based on two formative studies involving twelve experts. We evaluated the system with a sixteen within-subject study and found that Satori matches the performance of designer-created Wizard-of-Oz (WoZ) systems, without manual configurations or heuristics, thereby improving generalizability, reusability, and expanding the potential of AR assistance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。