通过建模工作记忆，让智能眼镜更懂用户心理状态，适时提供帮助。
ProMemAssist: Exploring Timely Proactive Assistance Through Working Memory Modeling in Multi-Modal Wearable Devices
- 用多模态信号实时建模用户工作记忆，模拟信息编码机制。
- 在12人实验中，帮助更精准，用户参与度高于大模型基线。
- 适合开发更体贴、懂用户认知负荷的主动式智能助手。
可穿戴AI系统旨在日常生活中提供及时协助，但现有方法多依赖用户主动触发或预设任务知识，忽视用户当前心理状态。我们提出ProMemAssist，一种基于智能眼镜的系统，利用多模态传感器信号实时建模用户的工作记忆（WM）。该系统基于工作记忆的认知理论，将感知信息表示为记忆项和情景片段，并引入如替换与干扰等编码机制。该工作记忆模型驱动一个时机预测器，平衡协助价值与打断成本。在12名参与者完成认知负荷较高的任务的用户研究中，ProMemAssist提供的协助更具选择性，用户参与度高于基于LLM的基线系统。定性反馈表明，工作记忆建模有助于实现更细腻、情境敏感的支持，为构建更关注用户状态的主动式代理提供了设计启示。
原文摘要 · Abstract (English)
Wearable AI systems aim to provide timely assistance in daily life, but existing approaches often rely on user initiation or predefined task knowledge, neglecting users' current mental states. We introduce ProMemAssist, a smart glasses system that models a user's working memory (WM) in real-time using multi-modal sensor signals. Grounded in cognitive theories of WM, our system represents perceived information as memory items and episodes with encoding mechanisms, such as displacement and interference. This WM model informs a timing predictor that balances the value of assistance with the cost of interruption. In a user study with 12 participants completing cognitively demanding tasks, ProMemAssist delivered more selective assistance and received higher engagement compared to an LLM baseline system. Qualitative feedback highlights the benefits of WM modeling for nuanced, context-sensitive support, offering design implications for more attentive and user-aware proactive agents.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。