用数学方法判断数据能否分段,让机器理解人类对数值的直觉。
Human-aligned Quantification of Numerical Data
- 基于轮廓系数和压缩信息量设计量化评估指标
- 轮廓系数>0.65且尖峰检验<0.5时可有效分段
- 轮廓系数更贴近人类对数值分类的直觉
量化数值数据面临两大挑战:一是判断数据是否适合自然分段,二是确定对应特定类别值的数值区间(称为“quantums”),这些区间代表统计上显著的状态。若可量化,连续数据流可转化为反映系统状态的“符号”序列。人们常凭常识或经验完成此任务,而信息论与计算机科学提供了可计算的度量方法。本研究评估了基于信息压缩与轮廓系数(Silhouette coefficient)的量化指标适用性,并考察其相互关联性及与“人类直觉”的一致性。结果表明,当轮廓系数高于0.65且尖峰检验(Dip Test)低于0.5时,数据可被有效分类;否则可视为单峰正态分布。此外,在可量化情形下,轮廓系数比源于信息压缩的“归一化质心距离”法更符合人类直觉。
原文摘要 · Abstract (English)
Quantifying numerical data involves addressing two key challenges: first, determining whether the data can be naturally quantified, and second, identifying the numerical intervals or ranges of values that correspond to specific value classes, referred to as "quantums," which represent statistically meaningful states. If such quantification is feasible, continuous streams of numerical data can be transformed into sequences of "symbols" that reflect the states of the system described by the measured parameter. People often perform this task intuitively, relying on common sense or practical experience, while information theory and computer science offer computable metrics for this purpose. In this study, we assess the applicability of metrics based on information compression and the Silhouette coefficient for quantifying numerical data. We also investigate the extent to which these metrics correlate with one another and with what is commonly referred to as "human intuition." Our findings suggest that the ability to classify numeric data values into distinct categories is associated with a Silhouette coefficient above 0.65 and a Dip Test below 0.5; otherwise, the data can be treated as following a unimodal normal distribution. Furthermore, when quantification is possible, the Silhouette coefficient appears to align more closely with human intuition than the "normalized centroid distance" method derived from information compression perspective.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。