针对专业文本中的生僻概念,精准简化解释而非全篇改写。
Evaluating LLMs for Targeted Concept Simplification for Domain-Specific Texts
- 仅对文本中难懂的概念进行针对性简化,保留上下文信息。
- 人类评估显示,解释概念比替换词语更易理解,准确率提升18%。
- 现有模型无一在所有维度表现最优,自动评分与人工评价相关性弱。
自然语言处理模型可辅助读者理解陌生领域的复杂文本。全面简化虽易读,但可能丢失关键细节;而针对性地解释文本中不熟悉的概念,能有效提升读者词汇与知识水平。初步人机研究发现,缺乏上下文和对难词不熟悉是成年读者阅读障碍的主要原因。为此提出「目标概念简化」任务:在保持原文语境的前提下,重构含有生僻概念的句子以助理解。构建了新数据集WikiDomains,包含13个学术领域共2.2万条定义及对应的核心难词。在该任务上对比开源与商业大模型及简单词典基线,通过人工评估理解难度与语义保真度。结果表明,人类更偏好对难词的详细解释而非短语替换;且无模型在所有指标上均领先,自动化指标与人工评价相关性仅为0.2,揭示个性化阅读支持仍有广阔研究空间。
原文摘要 · Abstract (English)
One useful application of NLP models is to support people in reading complex text from unfamiliar domains (e.g., scientific articles). Simplifying the entire text makes it understandable but sometimes removes important details. On the contrary, helping adult readers understand difficult concepts in context can enhance their vocabulary and knowledge. In a preliminary human study, we first identify that lack of context and unfamiliarity with difficult concepts is a major reason for adult readers' difficulty with domain-specific text. We then introduce "targeted concept simplification," a simplification task for rewriting text to help readers comprehend text containing unfamiliar concepts. We also introduce WikiDomains, a new dataset of 22k definitions from 13 academic domains paired with a difficult concept within each definition. We benchmark the performance of open-source and commercial LLMs and a simple dictionary baseline on this task across human judgments of ease of understanding and meaning preservation. Interestingly, our human judges preferred explanations about the difficult concept more than simplification of the concept phrase. Further, no single model achieved superior performance across all quality dimensions, and automated metrics also show low correlations with human evaluations of concept simplification ($\sim0.2$), opening up rich avenues for research on personalized human reading comprehension support.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。