文本本身具有内在曲率,可用来提升长文本推理与生成效果。
Text Has Curvature
- 提出Texture信号,基于上下文信念差定义词级曲率。
- 实证与理论证明语言语义推理非平面,存在固有曲率。
- 曲率引导压缩与路由,提升长文本生成与检索效果。
文本是否具有内在曲率?尽管语言常被建模在双曲空间或混合曲率流形中,但一个基本科学问题仍未解决:曲率对文本本身意味着什么,而非嵌入空间的产物?我们论证文本确实具有曲率,并提出可检测、定义与利用的方法。为此,我们提出Texture——一种文本原生的、词级离散曲率信号,贡献如下:(a) 存在性:通过实证与理论证明自然语料中的语义推理是非平坦的,即语言具有固有曲率;(b) 定义:通过施罗丁格桥统一左右文信念,定义曲率场,正值表示上下文聚焦语义,负值表示歧义扩散;(c) 实用性:Texture可作为通用度量与控制原语,无需几何训练即可实现几何调控;我们在两项代表性任务中验证其有效性:曲率引导压缩提升长上下文推理,曲率引导路由改进检索增强生成。结果确立了文本原生曲率范式,使曲率可测量且实用。
原文摘要 · Abstract (English)
Does text have an intrinsic curvature? Language is increasingly modeled in curved geometries - hyperbolic spaces for hierarchy, mixed-curvature manifolds for compositional structure - yet a basic scientific question remains unresolved: what does curvature mean for text itself, in a way that is native to language rather than an artifact of the embedding space we choose? We argue that text does indeed have curvature, and show how to detect it, define it, and use it. To this end, we propose Texture, a text-native, word-level discrete curvature signal, and make three contributions. (a) Existence: We provide empirical and theoretical certificates that semantic inference in natural corpora is non-flat, i.e. language has inherent curvature. (b) Definition: We define Texture by reconciling left- and right-context beliefs around a masked word through a Schrodinger bridge, yielding a curvature field that is positive where context focuses meaning and negative where it fans out into competing continuations. (c) Utility: Texture is actionable: it serves as a general-purpose measurement and control primitive enabling geometry without geometric training; we instantiate it on two representative tasks, improving long-context inference through curvature-guided compression and retrieval-augmented generation through curvature-guided routing. Together, our results establish a text-native curvature paradigm, making curvature measurable and practically useful.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。