为法律摘要设计多角色评估框架，提升专业与普通用户可读性
PersonaMatrix: A Recipe for Persona-Aware Evaluation of Legal Summarization
- 构建六类用户视角的评估框架，覆盖法律专家与公众
- 创建美国民权案件摘要数据集，含深度、可读性等维度变化
- 揭示不同用户对摘要优劣判断差异，指导AI系统优化
法律文书通常冗长密集，难以理解，不仅普通人难懂，专业人士也常感吃力。尽管自动化摘要有潜力提升法律知识获取效率，但现有评价方式忽略用户与利益相关者的多元需求。需开发兼顾律师所需技术深度与公众自诉查询易用性的工具。本文提出PersonaMatrix框架，从六类用户（含法律与非法律人士）视角评估摘要质量。同时构建一个控制变量的试点数据集，涵盖美国民事权利案例摘要，沿深度、可读性和程序细节等维度进行调整，并引入多样性-覆盖率指数（DCI），以揭示角色感知与无角色感知评价者在摘要优劣判断上的分歧。该研究推动法律AI摘要系统针对专业与非专业用户双向优化，有望扩大法律知识的可及性。代码与数据已在GitHub公开。
原文摘要 · Abstract (English)
Legal documents are often long, dense, and difficult to comprehend, not only for laypeople but also for legal experts. While automated document summarization has great potential to improve access to legal knowledge, prevailing task-based evaluators overlook divergent user and stakeholder needs. Tool development is needed to encompass the technicality of a case summary for a litigator yet be accessible for a self-help public researching for their lawsuit. We introduce PersonaMatrix, a persona-by-criterion evaluation framework that scores summaries through the lens of six personas, including legal and non-legal users. We also introduce a controlled dimension-shifted pilot dataset of U.S. civil rights case summaries that varies along depth, accessibility, and procedural detail as well as Diversity-Coverage Index (DCI) to expose divergent optima of legal summary between persona-aware and persona-agnostic judges. This work enables refinement of legal AI summarization systems for both expert and non-expert users, with the potential to increase access to legal knowledge. The code base and data are publicly available in GitHub.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。