发现话语结构决定信息密度波动，超越均匀信息理论
Surprise! Uniform Information Density Isn't the Whole Story: Predicting Surprisal Contours in Long-form Discourse
- 基于话语层级结构预测信息密度变化
- 深层嵌套结构比浅层结构更显著影响信息率
- 适合研究语言生成与认知机制的学者
均匀信息密度（UID）假说认为说话者倾向于在语言单位间平均分配信息以实现高效沟通。然而，文本和话语中的信息速率并非完全均匀。这些波动通常被视为对均匀目标的次要扰动，但另一种解释是：UID并非唯一调节语言信息内容的功能因素。说话者还可能为维持兴趣、遵循写作规范或构建有力论点而调整信息速率。本文提出一种新功能压力——结构化语境假说，即信息速率随话语的层级结构位置而调节。通过利用来自话语结构的预测因子，我们预测了从大规模语言模型中提取的自然话语的意外度轮廓。结果表明，层级结构预测因子显著影响话语的信息轮廓，且深度嵌套的结构预测因子比浅层结构更具预测力。该研究首次突破UID框架，提出了可验证的假设，解释为何信息速率以可预测方式波动。
原文摘要 · Abstract (English)
The Uniform Information Density (UID) hypothesis posits that speakers tend to distribute information evenly across linguistic units to achieve efficient communication. Of course, information rate in texts and discourses is not perfectly uniform. While these fluctuations can be viewed as theoretically uninteresting noise on top of a uniform target, another explanation is that UID is not the only functional pressure regulating information content in a language. Speakers may also seek to maintain interest, adhere to writing conventions, and build compelling arguments. In this paper, we propose one such functional pressure; namely that speakers modulate information rate based on location within a hierarchically-structured model of discourse. We term this the Structured Context Hypothesis and test it by predicting the surprisal contours of naturally occurring discourses extracted from large language models using predictors derived from discourse structure. We find that hierarchical predictors are significant predictors of a discourse's information contour and that deeply nested hierarchical predictors are more predictive than shallow ones. This work takes an initial step beyond UID to propose testable hypotheses for why the information rate fluctuates in predictable ways
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。