arXiv:2607.01391cs.LGcs.AI2026-07中稿 · ICML

探索医疗记录中数值编码的最佳方式,发现混合编码更实用可靠。

How Should Transformers Encode Numeric Values in Electronic Health Records?

论文配图:How Should Transformers Encode Numeric Values in Electronic Health Records?
图 1 · 摘自论文原文
  • 用分箱后投影的混合编码法保留数值信息
  • 分箱数量随数据集大小呈幂律增长,最优解可预测
  • 实际临床任务中稳健计算比精确算术更重要

如何在基于Transformer的序列处理中编码电子健康记录(EHR)中的数值?我们通过嵌入真实EHR数据的合成算术任务和真实的临床预测任务,系统比较了离散、连续及混合编码策略。研究揭示了数值精度、优化稳定性与架构灵活性之间的权衡。当架构允许时,显式建模值-概念交互的方法在对精度敏感的算术任务上表现最佳。而保留数值但先分箱再投影的混合令牌方法更具鲁棒性和普适性,最优分箱数随数据集规模呈现简单经验幂律。在各类任务中,模型始终表现出可靠的“足够好”数值计算能力,而非精确算术;临床收益取决于任务类型。这表明实践中稳健性与可部署性常优于极致精度,支持混合令牌方法作为默认实践。

原文摘要 · Abstract (English)

How do we encode numeric values in transformer-based sequence processing, particularly in electronic health record (EHR) data? We systematically compare discrete, continuous, and hybrid value encoding strategies using synthetic arithmetic tasks embedded within real-world EHR data, as well as real-world clinical prediction tasks. Our study reveals trade-offs between numeric precision, optimisation stability, and architectural flexibility. We find that approaches that explicitly model value-concept interactions perform best on precision-sensitive arithmetic tasks when architectural constraints permit. Hybrid token-based approaches that retain numeric values but apply binning prior to projection provide a more robust and broadly applicable alternative, with the optimal number of bins following a simple empirically derived power-law in dataset size. Across tasks, models consistently exhibit reliable "good enough" numeric computation rather than exact arithmetic, while clinical gains from incorporating laboratory values are task-dependent. This suggests that robustness and deployability often outweigh maximal numeric precision in practice, motivating hybrid token-based approaches as a practical default.

Transformer医疗记录数值编码混合方法

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。