用事实要点辅助法律人评估和改进AI生成的证词摘要
Supporting Humans in Evaluating AI Summaries of Legal Depositions
- 基于事实要点设计用户友好的评估工具
- 帮助法律人员判断两个摘要优劣，准确率提升显著
- 适合法律从业者与AI协作场景，提升摘要可靠性
大型语言模型（LLMs）在文档摘要中的应用日益广泛，但在法律领域，证词摘要的事实准确性至关重要。基于事实要点（nugget-based）的方法已被证明对自动化摘要评估非常有效。本文将此类方法应用于用户端，探索其如何直接支持终端用户。尽管已有系统展示了基于要点评估的潜力，但其对终端用户的实际支持作用仍待深入研究。聚焦法律领域，我们提出一个原型系统，采用基于事实要点的方法，协助法律专业人士在两个具体场景中工作：(1) 判断两个摘要哪个更优；(2) 手动改进自动生成的摘要。该系统通过提供可验证的事实单元，增强人类对AI摘要的信任与控制力。
原文摘要 · Abstract (English)
While large language models (LLMs) are increasingly used to summarize long documents, this trend poses significant challenges in the legal domain, where the factual accuracy of deposition summaries is crucial. Nugget-based methods have been shown to be extremely helpful for the automated evaluation of summarization approaches. In this work, we translate these methods to the user side and explore how nuggets could directly assist end users. Although prior systems have demonstrated the promise of nugget-based evaluation, its potential to support end users remains underexplored. Focusing on the legal domain, we present a prototype that leverages a factual nugget-based approach to support legal professionals in two concrete scenarios: (1) determining which of two summaries is better, and (2) manually improving an automatically generated summary.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。