实时主动提示对话，让助听设备悄悄帮你说好话。
LLAMAPIE: Proactive In-Ear Conversation Assistants
- 用小模型判断何时提醒，大模型生成简洁回应
- 用户实测偏好主动助手，比被动响应更受欢迎
- 专为耳机设备设计，本地运行不打扰对话
我们提出 LlamaPIE，首个通过可穿戴设备实时主动提供对话辅助的系统。与需主动唤起的传统语言模型不同，该助手在后台运行，无需打断即可预判需求并给出简明建议。针对响应时机判断、生成简洁回应、利用用户知识实现上下文感知、以及本地实时处理等挑战，我们构建了一个半合成对话数据集，并采用双模型架构：小型模型决定是否响应，大型模型生成回复内容。在真实数据集上的评估表明，该方法能有效提供有帮助且不突兀的辅助。在 Apple Silicon M2 硬件上实现的用户研究显示，参与者显著偏好主动型助手，优于无辅助基线和被动响应模型，验证了 LlamaPIE 提升实时对话的潜力。
原文摘要 · Abstract (English)
We introduce LlamaPIE, the first real-time proactive assistant designed to enhance human conversations through discreet, concise guidance delivered via hearable devices. Unlike traditional language models that require explicit user invocation, this assistant operates in the background, anticipating user needs without interrupting conversations. We address several challenges, including determining when to respond, crafting concise responses that enhance conversations, leveraging knowledge of the user for context-aware assistance, and real-time, on-device processing. To achieve this, we construct a semi-synthetic dialogue dataset and propose a two-model pipeline: a small model that decides when to respond and a larger model that generates the response. We evaluate our approach on real-world datasets, demonstrating its effectiveness in providing helpful, unobtrusive assistance. User studies with our assistant, implemented on Apple Silicon M2 hardware, show a strong preference for the proactive assistant over both a baseline with no assistance and a reactive model, highlighting the potential of LlamaPie to enhance live conversations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。