研究医疗事件建模中表示方式对预测效果的影响，发现融合编码可显著提升关键指标。
Representation Before Training: A Fixed-Budget Benchmark for Generative Medical Event Models
- 通过融合代码与数值信息构建输入表示，提升模型表达能力
- 融合表示使死亡率预测AUC提升至0.915，住院时长预测AUC达0.788
- 轻量级的CLIF映射保留性能且利于多中心应用，适合临床部署
生成式医疗事件模型的预测能力受限于临床事件的编码方式，但输入表示常被其他系统设计掩盖。本研究在统一的一轮预训练预算下，评估不同表示策略对下游30项临床结果的影响。在MIMIC-IV数据集上训练28个匹配的Transformer模型，开展三项实验：（1）量化粒度、参考范围锚定与代码-值融合；（2）值编码方式（硬分箱、软离散化、归一化xVal）与时间编码（事件顺序、时间标记、入院相对RoPE）组合；（3）原始MIMIC实验室/生命体征代码与压缩保持扰动的通用纵向重症格式（CLIF）映射代码对比。实验1中，融合编码将死亡率AUROC从0.891提升至0.915（BH校正p<0.001），住院时长AUROC从0.763升至0.788（BH校正p<0.001），13项回归任务平均斯皮尔曼相关性从0.414升至0.494。三种时间编码中，事件顺序与入院相对RoPE平均表现不劣于插入时间标记，同时序列长度缩短11%。CLIF映射在单中心设置下保持下游性能，且生成更小、更具临床可解释性的词表，适配多中心使用。细粒度量化、参考范围锚定和软离散化在特定任务中有效，而归一化xVal表现持续低于离散与软离散方案，体现近中位数抑制现象。
原文摘要 · Abstract (English)
Every prediction from a generative medical event model is bounded by how clinical events are tokenized, yet input representation is rarely isolated from other system and architectural choices. We evaluate how representation decisions affect downstream prediction after a shared one-epoch pretraining budget. We train 28 matched transformers on MIMIC-IV and evaluate them on 30 clinical outcomes in three experiments: (1) quantization granularity, reference-range anchoring, and code-value fusion; (2) value encoding (hard bins, soft discretization, code-normalized xVal) crossed with temporal encoding (event order, time tokens, admission-relative RoPE); and (3) native MIMIC laboratory/vital codes versus the Common Longitudinal ICU Format (CLIF)-remapped laboratory/vital codes with compression-preserving perturbation arms. In Experiment 1, fused code-value tokenization improves mortality AUROC from 0.891 to 0.915 (BH-adjusted p < 0.001), hospital length-of-stay AUROC from 0.763 to 0.788 (BH-adjusted p < 0.001), and, for the decile fused-vs-unfused comparison, mean regression Spearman rho across the 13 regression outcomes from 0.414 to 0.494. Across the three temporal encodings, event order only and admission-relative RoPE match or exceed inserting time tokens on average while shortening sequences by 11%. CLIF remapping preserves downstream performance in our single-site setting while yielding a smaller, clinically interpretable token set compatible with multi-site use. Finer-than-decile quantization, reference-range anchoring, and soft discretization help in selective outcomes, while code-normalized xVal remains well below the discrete and soft families, consistent with near-median suppression that persists after the affine variant.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。