测试大模型在信息压缩和对抗攻击下的知识可靠性,发现规模不是可靠性的保证。
The Drill-Down and Fabricate Test (DDFT): A Protocol for Measuring Epistemic Robustness in Language Models
- 设计双系统模型:生成流畅文本 + 验证事实准确性的认知机制
- 9个前沿模型在8个领域测试,发现模型大小与可靠性无关(相关性不显著)
- 小模型也能表现稳健,适合对可靠性要求高的实际部署场景
现有语言模型评估仅考察理想条件下的知识掌握程度,却无法衡量其在真实压力下的知识可靠性。静态基准如MMLU和TruthfulQA无法区分缺乏知识的模型与验证机制在信息退化或对抗攻击下崩溃的模型。本文提出钻探与虚构测试(DDFT),用于测量认知鲁棒性:模型在逐步语义压缩和对抗性伪造下的事实准确性保持能力。我们构建了包含语义系统(生成流畅文本)和认知验证器(验证事实准确性)的双系统认知模型。基于对9个前沿模型在8个知识领域、5个压缩层级(共1,800次逐轮评估)的测试发现,认知鲁棒性与传统设计范式正交。参数量(r=0.083, p=0.832)和架构类型(r=0.153, p=0.695)均不能显著预测鲁棒性,表明其源于训练方法和验证机制,而非当前主流路径。错误检测能力与整体鲁棒性高度负相关(rho=-0.817, p=0.007),是关键瓶颈。旗舰模型虽规模庞大但仍脆弱,而较小模型可实现稳健表现,挑战了规模即可靠的假设。DDFT框架为关键应用部署前评估认知鲁棒性提供了理论基础与实践工具。
原文摘要 · Abstract (English)
Current language model evaluations measure what models know under ideal conditions but not how robustly they know it under realistic stress. Static benchmarks like MMLU and TruthfulQA cannot distinguish a model that lacks knowledge from one whose verification mechanisms collapse when information degrades or adversaries probe for weaknesses. We introduce the Drill-Down and Fabricate Test (DDFT), a protocol that measures epistemic robustness: a model's ability to maintain factual accuracy under progressive semantic compression and adversarial fabrication. We propose a two-system cognitive model comprising a Semantic System that generates fluent text and an Epistemic Verifier that validates factual accuracy. Our findings, based on evaluating 9 frontier models across 8 knowledge domains at 5 compression levels (1,800 turn-level evaluations), reveal that epistemic robustness is orthogonal to conventional design paradigms. Neither parameter count (r=0.083, p=0.832) nor architectural type (r=0.153, p=0.695) significantly predicts robustness, suggesting it emerges from training methodology and verification mechanisms distinct from current approaches. Error detection capability strongly predicts overall robustness (rho=-0.817, p=0.007), indicating this is the critical bottleneck. We find that flagship models exhibit brittleness despite their scale, while smaller models can achieve robust performance, challenging assumptions about the relationship between model size and reliability. The DDFT framework provides both theoretical foundation and practical tools for assessing epistemic robustness before deployment in critical applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。