用大样本互为零假设，自动验证大模型提出的猜想是否可信。
NxN E-valuation: Hypothesis Certification via a Conformal CRT Null
- 利用数据集内样本互作零假设，实现无需定制流程的假设验证。
- 在大模型生成的猜想中，有效识别并排除幻觉内容。
- 适合依赖大模型提假设但怕出错的探索性系统使用。
我们提出NxN E-valuation，一种基于e-value的假设认证算法，可在拥有足够大数据集的前提下，无需构建特定于案例的认证流程（如专用零假设），即可验证假设。该方法特别适用于基于大语言模型（LLM）的探索系统，其中大模型擅长提出假设但存在严重幻觉问题，导致其输出无法直接使用。现有解决方法如自验证、留出测试等均存在缺陷：自验证存在循环论证，留出测试可能因偶然相关性通过虚假假设。NxN E-valuation 利用训练数据本身，让不同样本互为零假设，直接实现条件随机化检验（CRT），对每个假设进行认证。只要大模型生成的内容是针对单个样本的有效假设，该方法即可成为比自验证和留出测试更优的通用替代方案。
原文摘要 · Abstract (English)
We propose NxN E-valuation, a handy, e-value-based hypothesis-certification algorithm that lets a hypothesis be verified without building any case-specific certification procedure---such as constructing a dedicated null hypothesis---as long as a large enough dataset is available. The method is especially suited to LLM-based exploration systems, where LLMs are remarkably good at proposing hypotheses but suffer badly from hallucination; this hallucination prevents us from harvesting LLM outputs directly, and existing remedies each fall short. The most common solutions include letting the LLM verify or correct itself circular verification and held-out testing (where false hypotheses can still pass via spurious correlations), among other remedies detailed in the introduction. To resolve this, NxN E-valuation exploits the naturally existing large training set and lets different samples serve as null hypotheses for one another. This design directly realizes a conditional randomization test (CRT) that certifies each hypothesis. The approach can be a universally better replacement for at least LLM circular verification and held-out-data testing, provided the LLM's generations are hypotheses that apply to each individual sample.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。