arXiv:2506.21582cs.CLcs.AI2025-06被引 1

让新手也能用智能代理完成高级文本分析,三步自动搞定。

VIDEE: Visual and Interactive Decomposition, Execution, and Evaluation of Text Analytics with Intelligent Agents

  • 人机协同分解任务,用蒙特卡洛树搜索结合用户反馈生成分析思路。
  • 自动生成可执行分析流程,并通过LLM评估与可视化验证结果。
  • 适合无NLP基础的分析师,尤其帮助初学者突破技术门槛。

传统文本分析依赖专业的自然语言处理知识,对初级分析师构成障碍。大语言模型(LLM)的发展使文本分析更易访问和自动化(如主题检测、摘要生成、信息抽取等)。我们提出VIDEE系统,支持初级数据分析师通过智能代理进行高级文本分析。VIDEE采用人机协同工作流,包含三个阶段:(1) 分解,利用人机协作的蒙特卡洛树搜索算法实现生成式推理并融入用户反馈;(2) 执行,生成可运行的文本分析流水线;(3) 评估,集成基于LLM的评估与可视化,辅助用户验证结果。通过两项定量实验评估系统有效性,并分析常见代理错误。一项涵盖从零经验到专家级别的用户研究显示系统可用性良好,并揭示不同用户的行为模式。研究发现为人类-代理协作设计提供了启示,验证了VIDEE对非专家用户的实用性,并为未来智能文本分析系统优化提供依据。

原文摘要 · Abstract (English)

Text analytics has traditionally required specialized knowledge in Natural Language Processing (NLP) or text analysis, which presents a barrier for entry-level analysts. Recent advances in large language models (LLMs) have changed the landscape of NLP by enabling more accessible and automated text analysis (e.g., topic detection, summarization, information extraction, etc.). We introduce VIDEE, a system that supports entry-level data analysts to conduct advanced text analytics with intelligent agents. VIDEE instantiates a human-agent collaroration workflow consisting of three stages: (1) Decomposition, which incorporates a human-in-the-loop Monte-Carlo Tree Search algorithm to support generative reasoning with human feedback, (2) Execution, which generates an executable text analytics pipeline, and (3) Evaluation, which integrates LLM-based evaluation and visualizations to support user validation of execution results. We conduct two quantitative experiments to evaluate VIDEE's effectiveness and analyze common agent errors. A user study involving participants with varying levels of NLP and text analytics experience -- from none to expert -- demonstrates the system's usability and reveals distinct user behavior patterns. The findings identify design implications for human-agent collaboration, validate the practical utility of VIDEE for non-expert users, and inform future improvements to intelligent text analytics systems.

文本分析智能代理人机协作

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。