arXiv:2606.29605cs.CL2026-06

90%的LLM生成临床数据是重复的,真实新内容仅占10.9%。

How much of an LLM-generated clinical corpus is actually new? A production-scale measurement of content redundancy for provenance classification

论文配图:How much of an LLM-generated clinical corpus is actually new? A production-scale measurement of content redundancy for provenance classification
图 1 · 摘自论文原文
  • 通过溯源分解法逐标记每条文本来源,发现冗余率高达79.4%
  • 冗余主要来自原文复制和跨记录重复,仅前者可无损去除
  • 去重后模型性能提升,证明冗余影响训练效果

临床机器学习越来越多依赖大语言模型(LLMs)生成的语料库,而非由临床医生标注。本文分析了由多智能体临床提取管道处理167,034名患者叙述所生成的25.1亿个令牌,覆盖十一通道中的十个文本通道。提出基于溯源的冗余分解方法,对全部输出进行逐令牌源分类。结果显示,仅10.9%的内容为可训练独有信息,79.4%为冗余;原始令牌数量使信息量被夸大近九倍。冗余由两种机制导致:一是源上下文直接复制至每项字段,二是生成文本在记录间重复,仅前者可无损移除。独立的、无需模型的压缩分析验证了该结果,识别出相同机制。其中一个通道几乎无冗余,表明冗余程度取决于各通道设计,而非固定属性。未校正的冗余会放大长且复杂的病历生成的样本数,从而扭曲语料库的令牌级训练分布。在控制下游测试中,去重后在相等令牌预算下,临床编码器在外部疾病识别基准上表现更优,且在不同微调深度下均稳健,另一基准也复现成功。分类工具已开源。

原文摘要 · Abstract (English)

Clinical machine learning increasingly relies on training corpora generated by large language models (LLMs) rather than annotated by clinicians, and such corpora are described and reused largely on the basis of their reported scale. We test whether volume reflects information content. Analysing the complete output of a multi-agent clinical extraction pipeline applied to 167,034 patient narratives, 2.51 billion generated tokens across the ten text-bearing channels of an eleven-channel pipeline, we introduce Provenance-based Redundancy Decomposition, a token-level classification of the entire output by source. Only 10.9% of the output is trainable-unique content while 79.4% is redundant; raw token count overstates information content by roughly ninefold. The redundancy arises through two distinct mechanisms, verbatim copying of source context into per-item fields, and duplication of generated text across records, of which only the former is losslessly removable. An independent, model-free analysis based on lossless compression confirms the redundancy, recovering the two mechanisms without reference to the provenance labels. One pipeline channel carries almost no redundancy, showing that the level of redundancy depends on how each channel is structured rather than being a fixed property of LLM extraction. Because uncorrected redundancy up-weights the longer, more complex presentations that generate the most items, it skews the token-level training distribution of the corpus, a property we measure directly. In a controlled downstream test, de-duplicating the corpus before adaptation improved a clinical encoder on external disease-recognition benchmarks at equal token budget, robustly across adaptation depths and replicated on a second benchmark, confirming that the redundancy carries a measurable cost beyond storage. The classification tool is released openly.

LLM生成临床数据去重冗余分析

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。