arXiv:2605.07068cs.CLcs.AI2026-05被引 4

提出迭代式知识编译框架WiCER,解决大模型维基知识丢失问题。

WiCER: Wiki-memory Compile, Evaluate, Refine Iterative Knowledge Compilation for LLM Wiki Systems

  • 基于诊断反馈迭代优化知识编译,主动找回遗漏关键事实。
  • 1-2轮迭代恢复80%丢失性能,灾难性失败率降低55%。
  • 适合构建高可靠、低延迟的维基知识系统,提升推理准确性。

LLM维基模式通过将领域知识编译为持久化文档,并以键值缓存推理方式提供给大模型,可实现亚秒级延迟且零检索失败。但实际应用面临编译差距:大模型在提炼原始文档生成维基时,可能灾难性地丢弃关键事实。我们在17个RepLiQA领域(6,800个问题)中评估发现,全上下文键值缓存推理在小规模下优于RAG(4.38 vs. 4.08分,TTFT快7.3倍),但在大规模下因注意力稀释而劣于RAG;盲编译完全失败(得分2.14–2.32,灾难性失败率53%–60%)。为此,我们提出受反例引导抽象精化(CEGAR)启发的迭代算法WiCER,通过诊断探针检测编译维基中的遗漏事实,并在后续编译中强制保留。1-2轮迭代即可恢复80%损失质量(平均得分3.24 vs. 原始全上下文3.47),相较基线减少55%灾难性失败。跨全部17个领域的消融实验表明,针对性诊断(+0.95)带来主要增益,而非通用固定(+0.16)。所有代码与基准均已开源,支持可复现研究。

原文摘要 · Abstract (English)

The LLM Wiki pattern, to compile and provide domain knowledge into a persistent artifact and serve it to LLMs via KV cache inference, promises context access at sub-second latency with zero retrieval failure. Realizing this requires solving the compilation gap: LLM compilation distilling raw documents into a wiki without catastrophically discarding critical facts. We characterize this gap across 17 RepLiQA domains (6,800 questions): we observe that full context KV cache inference outperforms RAG on curated knowledge (4.38 vs. 4.08 out of 5, 7.3 faster TTFT) but degrades below RAG at scale due to attention dilution, and blind compilation fails entirely (2.14 to 2.32 vs. 3.46, 53 to 60% catastrophic failure rate). To address the compilation gap, we propose WiCER (Wiki-memory Compile, Evaluate, Refine), an iterative algorithm inspired by counterexample-guided abstraction refinement (CEGAR) that closes this gap. WiCER evaluates compiled wikis against diagnostic probes, identifies dropped facts, and forces their preservation in subsequent compilations. One to two iterations recover 80% of lost quality (mean 3.24 vs. 3.47 for raw full-context across the 15 topics with baselines), reducing catastrophic failures by 55% relative. An ablation across all 17 topics confirms that targeted diagnosis (+0.95), not generic pinning (+0.16), drives the gains. All code and benchmarks are released for reproducible research.

知识编译大模型维基系统迭代优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。