用大模型生成个性化解释,让AR应用更懂用户、更可信。
PILAR: Personalizing Augmented Reality Interactions with LLM-based Human-Centric and Trustworthy Explanations for Daily Use Cases
- 用预训练大模型动态生成符合上下文的个性化解释
- 用户任务完成速度提升40%,满意度与透明感显著增强
- 适合希望提升AR交互体验的开发者与产品设计者
AI驱动的增强现实(AR)系统正日益融入日常生活,随之对实时用户交互可解释性的需求也日益增长。传统可解释AI(XAI)方法多依赖特征或示例生成解释,难以提供动态、情境化、个性化的以人为本洞察,且常分头处理‘何时、何事、如何’等解释维度,导致体验割裂、不够自然。为此,我们提出PILAR框架,利用预训练大语言模型(LLM)生成上下文感知、个性化解释,实现真实场景中更直观、可信的实时交互体验。该框架采用统一的LLM方法,而非多种技术分别应对不同解释维度,能根据用户需求动态调整解释内容,增强信任与参与度。我们在一个真实世界的应用(如个性化食谱推荐)中实现了PILAR原型,集成实时物体检测、食谱推荐及基于用户饮食偏好的LLM解释功能。通过16名参与者完成的用户研究,对比了基于LLM的解释界面与传统模板式界面,结果显示:使用LLM界面的用户任务完成速度快40%,并报告更高的满意度、易用性和感知透明度。
原文摘要 · Abstract (English)
Artificial intelligence (AI)-driven augmented reality (AR) systems are becoming increasingly integrated into daily life, and with this growth comes a greater need for explainability in real-time user interactions. Traditional explainable AI (XAI) methods, which often rely on feature-based or example-based explanations, struggle to deliver dynamic, context-specific, personalized, and human-centric insights for everyday AR users. These methods typically address separate explainability dimensions (e.g., when, what, how) with different explanation techniques, resulting in unrealistic and fragmented experiences for seamless AR interactions. To address this challenge, we propose PILAR, a novel framework that leverages a pre-trained large language model (LLM) to generate context-aware, personalized explanations, offering a more intuitive and trustworthy experience in real-time AI-powered AR systems. Unlike traditional methods, which rely on multiple techniques for different aspects of explanation, PILAR employs a unified LLM-based approach that dynamically adapts explanations to the user's needs, fostering greater trust and engagement. We implement the PILAR concept in a real-world AR application (e.g., personalized recipe recommendations), an open-source prototype that integrates real-time object detection, recipe recommendation, and LLM-based personalized explanations of the recommended recipes based on users' dietary preferences. We evaluate the effectiveness of PILAR through a user study with 16 participants performing AR-based recipe recommendation tasks, comparing an LLM-based explanation interface to a traditional template-based one. Results show that the LLM-based interface significantly enhances user performance and experience, with participants completing tasks 40% faster and reporting greater satisfaction, ease of use, and perceived transparency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。