提出新评估方法SciGisPy，更好衡量医学文本简化是否保留核心含义。
SciGisPy: a Novel Metric for Biomedical Text Simplification via Gist Inference Score
- 基于模糊痕迹理论改进摘要推断评分，结合领域术语处理。
- 在Cochrane数据集上正确识别简化文本率达84%，远超原版的44.8%。
- 适合评估医学文本简化质量，尤其关注核心意义保留。
医学文献常用高度专业化的语言，非专业人士理解困难。自动文本简化（ATS）可提升可读性并保留关键信息，但现有评估方法存在局限。通用指标如SARI、BLEU、ROUGE仅关注表面特征，可读性指标如FKGL和ARI无法反映领域术语或核心含义（概要）的传递效果。为此，我们提出SciGisPy，一种受模糊痕迹理论（FTT）中概要推断评分（GIS）启发的新评估指标。该方法通过语义分块、信息熵理论及专用嵌入等增强手段，重构了适应医学领域的GIS，并移除不适用指标。在Cochrane医学文本简化数据集上的实验表明，SciGisPy显著优于原始GIS，正确识别简化文本率从44.8%提升至84%。消融研究进一步验证其更准确捕捉医学内容本质意义的能力。
原文摘要 · Abstract (English)
Biomedical literature is often written in highly specialized language, posing significant comprehension challenges for non-experts. Automatic text simplification (ATS) offers a solution by making such texts more accessible while preserving critical information. However, evaluating ATS for biomedical texts is still challenging due to the limitations of existing evaluation metrics. General-domain metrics like SARI, BLEU, and ROUGE focus on surface-level text features, and readability metrics like FKGL and ARI fail to account for domain-specific terminology or assess how well the simplified text conveys core meanings (gist). To address this, we introduce SciGisPy, a novel evaluation metric inspired by Gist Inference Score (GIS) from Fuzzy-Trace Theory (FTT). SciGisPy measures how well a simplified text facilitates the formation of abstract inferences (gist) necessary for comprehension, especially in the biomedical domain. We revise GIS for this purpose by introducing domain-specific enhancements, including semantic chunking, Information Content (IC) theory, and specialized embeddings, while removing unsuitable indexes. Our experimental evaluation on the Cochrane biomedical text simplification dataset demonstrates that SciGisPy outperforms the original GIS formulation, with a significant increase in correctly identified simplified texts (84% versus 44.8%). The results and a thorough ablation study confirm that SciGisPy better captures the essential meaning of biomedical content, outperforming existing approaches.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。