动态更新知识库，让大模型实时捕捉重要信息变化。
Streaming Knowledge Compilation: Proactive Materiality-Scored Pinning for Time-Evolving LLM Wikis
- 用动态重要性评分预判关键信息，提前存入缓存。
- 金融与维基场景下，知识更新效果显著优于传统方法。
- 适合需要实时知识演化的系统开发者和研究者。
大型语言模型知识库系统通过预填充键值缓存实现高效推理，但假设知识库静态不变——这一假设在信息持续演进时失效。本文提出流式知识编译：给定文档流、固定令牌预算及未知的未来查询，在编译知识库时最小化累积后悔值，对比具备完美预见力的离线最优解。核心思想是引入材料度信号ϕ_t(k,n)∈[0,1]，对实体k在时间t的重要性进行评分，作为查询相关性的代理，用于主动预存内容；理论证明其后悔值为O(√(T log K))，其中ε=𝔼[|ϕ_t−ϕ̂_t|]为唯一领域相关参数。在金融领域，ϕ_t由冻结的Llama 3.1 8B分类头预测异常股价波动（76K文章严格时序划分，AUROC=0.728），被预测为高重要性的文章实际波动率高出1.49倍；在维基百科中，ϕ_t采用异常编辑比率（AER），即跨样本标准化的编辑速度，表明该方法可跨领域通用。在173对金融与119对维基的端到端问答测试中，揭示了后训练知识导致的‘大模型作为评判者’偏差，证明后悔值分析才是评估知识编译系统可靠指标。金融累计后悔收敛至-20.0（每步-0.12）；维基百科为+16.0（每步+0.13），正值确认维基编辑内容确为后训练生成，更丰富上下文持续提升表现（无维基3.80 vs. 优化器4.74），消除该偏差。该O(√(T log K))保证适用于任何能从流信号预测知识缺口的领域。
原文摘要 · Abstract (English)
LLM wiki systems compile knowledge into pre-filled KV caches for efficient inference, but assume a static corpus -- an assumption that fails whenever the underlying information landscape evolves. We formalize Streaming Knowledge Compilation: given a document stream, a fixed token budget, and future queries unknown at ingestion time, maintain a compiled wiki that minimizes cumulative regret against an offline oracle with perfect foresight. The enabling insight is a materiality signal $ϕ_t(k,n)\in[0,1]$ that scores document importance for entity $k$ at time $t$, acting as a query-relevance surrogate for proactive pinning before queries arrive; we prove an $O(\sqrt{T\log K})$ regret bound where $\varepsilon=\mathbb{E}[|ϕ_t-\hatϕ_t|]$ is the only domain-specific quantity. We instantiate in two domains: finance, where $ϕ_t$ is abnormal stock volatility predicted by frozen Llama 3.1 8B classification head (AUROC = 0.728 on 76K articles, strict temporal split; $1.49\times$ higher realized forward volatility for predicted-material articles); and Wikipedia, where $ϕ_t$ is the Abnormal Edit Ratio (AER), a cross-sectionally normalized edit velocity -- showing the same algorithm generalizes beyond the finance domain. End-to-end QA evaluation on 173 matched pairs (finance) and 119 (Wikipedia) reveals a pervasive LLM-as-judge confound on post-training knowledge, establishing that regret analysis -- not absolute QA scores -- is the reliable evaluation metric for compiled knowledge systems. Finance cumulative regret converges to -20.0 (-0.12/step); Wikipedia to +16.0 (+0.13/step), with the positive sign confirming that Wikipedia edit content is genuinely post-training -- richer context consistently improves scores (No Wiki 3.80 vs. Oracle 4.74) -- and eliminates this confound. The $O(\sqrt{T\log K})$ guarantee applies to any domain where knowledge gaps can be predicted from streaming signals.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。