用机器学习与生成AI高效分析用户反馈,提升产品体验评估效率
Integrating Multi-Label Classification and Generative AI for Scalable Analysis of User Feedback
- 用监督学习给用户评论打预设主题标签,实现快速分类
- 生成AI自动生成反馈摘要,助力管理层快速理解关键问题
- 发现情感分析不能准确反映用户满意度,需独立调查
在竞争激烈的软件市场中,用户体验(UX)评估对保障软件质量和实现长期产品成功至关重要。这类评估通常结合标准化问卷的量化指标与开放题收集的定性反馈。尽管开放题反馈能提供改进洞察并解释量化结果,但分析大量用户评论仍具挑战性且耗时。本文介绍了一家大型软件公司在长期用户体验测量项目中开发的技术,用于高效处理和解读大量用户评论。为提供评论的宏观概览,我们采用监督机器学习方法,为每条评论分配有意义的预定义主题标签。此外,我们展示了如何利用生成式AI(GenAI)创建简洁、信息丰富的反馈摘要,促进组织内部尤其是高层管理的成果传达。最后,我们探究用户评论中的情感是否可作为整体产品满意度的指标。结果表明,仅靠情感分析无法可靠反映用户满意度,产品满意度需通过专门调查来衡量用户对产品的感知。
原文摘要 · Abstract (English)
In highly competitive software markets, user experience (UX) evaluation is crucial for ensuring software quality and fostering long-term product success. Such UX evaluations typically combine quantitative metrics from standardized questionnaires with qualitative feedback collected through open-ended questions. While open-ended feedback offers valuable insights for improvement and helps explain quantitative results, analyzing large volumes of user comments is challenging and time-consuming. In this paper, we present techniques developed during a long-term UX measurement project at a major software company to efficiently process and interpret extensive volumes of user comments. To provide a high-level overview of the collected comments, we employ a supervised machine learning approach that assigns meaningful, pre-defined topic labels to each comment. Additionally, we demonstrate how generative AI (GenAI) can be leveraged to create concise and informative summaries of user feedback, facilitating effective communication of findings to the organization and especially upper management. Finally, we investigate whether the sentiment expressed in user comments can serve as an indicator for overall product satisfaction. Our results show that sentiment analysis alone does not reliably reflect user satisfaction. Instead, product satisfaction needs to be assessed explicitly in surveys to measure the user's perception of the product.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。