小规模语言单元即可解释概率缩减现象

Less is more: Probabilistic reduction is best explained by small-scale predictability measures

• 用n-gram代替完整语句作为认知规划单位
• 发现小规模预测指标已足够解释概率缩减
• 适合研究语言模型与认知关联的学者

本文聚焦于语言模型概率与认知现象关系研究中所需上下文量的界定问题。探究完整话语是否为观察概率缩减所必需，结果表明n-gram表示作为认知规划单位已足够，无需整句上下文。该发现支持在建模语言生成时采用更小尺度的预测机制，有助于提升模型对人类语言规划行为的理解精度。

原文摘要 · Abstract (English)

The primary research questions of this paper center on defining the amount of context that is necessary and/or appropriate when investigating the relationship between language model probabilities and cognitive phenomena. We investigate whether whole utterances are necessary to observe probabilistic reduction and demonstrate that n-gram representations suffice as cognitive units of planning.

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。