arXiv:2507.05973cs.CL2025-07被引 21
呼吁NLP领域重视真实世界影响评估，而非只关注指标表现。
We Should Evaluate Real-World Impact
- 分析ACL论文发现仅0.1%涉及真实影响评估
- 多数研究虽提影响但描述粗略，仍以指标为主
- 强调真实影响评估可加速技术落地与应用
ACL学术界对自然语言处理系统的真实世界影响评估兴趣甚微。对ACL文集的结构化调查显示，其中约0.1%的论文包含此类评估；且大多数提及影响评估的论文也仅作简略描述，仍以指标评估为核心。若能认真理解并评估NLP技术的真实影响，该技术将更具实用性，并更快被采纳和应用。
原文摘要 · Abstract (English)
The ACL community has very little interest in evaluating the real-world impact of NLP systems. A structured survey of the ACL Anthology shows that perhaps 0.1% of its papers contain such evaluations; furthermore most papers which include impact evaluations present them very sketchily and instead focus on metric evaluations. NLP technology would be more useful and more quickly adopted if we seriously tried to understand and evaluate its real-world impact.
NLP影响评估应用落地
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。