语言模型比较带单位的数量时依赖特定数字和单位的启发式策略。
Language Models Compare Quantities Using Number-specific and Unit-specific Heuristics
- 通过数值差和单位尺度差构建线性代理模型预测模型偏好
- 在比较边界附近准确率下降，错误具有系统性
- 适合研究模型推理机制或对齐偏差的学者
带有测量单位的数量（如110 cm和1.2 m）要求语言模型将数值与符号单位尺度结合。本文在涵盖多个单位系统的受控场景中研究语言模型如何比较此类数量。发现模型在比较边界附近准确率下降，微小的数值变化即决定正确答案。错误呈现系统性：线性代理模型可基于数值差和单位尺度差预测模型偏好，对相关子空间进行因果干预会改变模型输出。结果表明，语言模型并非先将表达式转换为统一量纲再比较，而是依赖数值和单位的特定启发式规则进行判断。
原文摘要 · Abstract (English)
Quantities with measurement units, such as 110 cm and 1.2 m, require language models (LMs) to combine a numeral with a symbolic unit scale. Here, we study how LMs compare such quantities in controlled settings spanning several unit systems. We find that accuracy degrades near the comparison boundary, where small changes in value determine the correct answer. The resulting errors are systematic: linear surrogate models predict LM preferences from numerical-difference and unit-scale-difference cues, and causal interventions on subspaces aligned with these variables shift model's output. The results suggest that LMs compare quantities through a bag of heuristics over numerals and units, rather than first converting both expressions to an exact shared-scale representation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。