arXiv:2510.23635cs.LGcs.AI2025-10

用真实用户反馈优化个人助理数据清洗,降低标注负担并提升质量。

Help the machine to help you: an evaluation in the wild of egocentric data cleaning via skeptical learning

  • 让用户实时修正标签,结合主动与被动数据验证标注准确性。
  • 四周期实验显示,该方法可减少标注工作量,同时提高数据质量。
  • 适合关注真实场景下智能助手数据可信度的研究者与开发者。

任何数字个人助理,无论用于支持任务执行、回答问题或管理日常事务(如健身计划),都需要高质量的标注数据才能正常运作。然而,用户主动标注或通过上下文推断(如手机传感器数据)生成的标注往往存在错误和噪声。以往关于怀疑学习(Skeptical Learning, SKEL)的研究通过对比离线主动标注与被动数据,评估了标注准确性,但未纳入最终用户对自身情境的确认,而用户才是自己上下文的最佳判断者。本研究在真实环境中评估SKEL的表现,让实际用户通过iLog移动应用在四周时间内根据当前视角和需求修正输入标签。结果表明,在用户投入与数据质量之间需权衡,但使用SKEL能显著减少标注负担,并提升收集数据的质量。

原文摘要 · Abstract (English)

Any digital personal assistant, whether used to support task performance, answer questions, or manage work and daily life, including fitness schedules, requires high-quality annotations to function properly. However, user annotations, whether actively produced or inferred from context (e.g., data from smartphone sensors), are often subject to errors and noise. Previous research on Skeptical Learning (SKEL) addressed the issue of noisy labels by comparing offline active annotations with passive data, allowing for an evaluation of annotation accuracy. However, this evaluation did not include confirmation from end-users, the best judges of their own context. In this study, we evaluate SKEL's performance in real-world conditions with actual users who can refine the input labels based on their current perspectives and needs. The study involves university students using the iLog mobile application on their devices over a period of four weeks. The results highlight the challenges of finding the right balance between user effort and data quality, as well as the potential benefits of using SKEL, which include reduced annotation effort and improved quality of collected data.

数据清洗用户反馈智能助手真实场景

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。