用AI自动生成大模型不确定性量化方法，效果优于人工设计。
Evolutionary Search for Automated Design of Uncertainty Quantification Methods
- 用大模型驱动的进化搜索自动生成Python代码形式的不确定量化方法。
- 在9个数据集上相对提升6.7%的ROC-AUC，且对分布外数据泛化能力强。
- 不同大模型演化出不同策略，部分模型复杂度提升反而性能下降。
大语言模型的不确定性量化（UQ）方法主要依赖领域知识和启发式设计，限制了其可扩展性和通用性。本文采用大模型驱动的进化搜索，自动发现以Python程序形式表示的无监督UQ方法。在原子命题验证任务中，所生成的方法优于强基准的手动设计方法，在9个数据集上实现了最高6.7%的相对ROC-AUC提升，并表现出稳健的分布外泛化能力。定性分析显示，不同大模型采用截然不同的演化策略：Claude系列模型持续生成高特征数线性估计器，而Gpt-oss-120B则偏好更简单、可解释的位置加权方案。令人惊讶的是，仅Sonnet 4.5和Opus 4.5能通过增加方法复杂度提升性能——而Opus 4.6相较前代反而出现性能退化。总体表明，大模型驱动的进化搜索是自动化、可解释幻觉检测器设计的有前景范式。
原文摘要 · Abstract (English)
Uncertainty quantification (UQ) methods for large language models are predominantly designed by hand based on domain knowledge and heuristics, limiting their scalability and generality. We apply LLM-powered evolutionary search to automatically discover unsupervised UQ methods represented as Python programs. On the task of atomic claim verification, our evolved methods outperform strong manually-designed baselines, achieving up to 6.7% relative ROC-AUC improvement across 9 datasets while generalizing robustly out-of-distribution. Qualitative analysis reveals that different LLMs employ qualitatively distinct evolutionary strategies: Claude models consistently design high-feature-count linear estimators, while Gpt-oss-120B gravitates toward simpler and more interpretable positional weighting schemes. Surprisingly, only Sonnet 4.5 and Opus 4.5 reliably leverage increased method complexity to improve performance -- Opus 4.6 shows an unexpected regression relative to its predecessor. Overall, our results indicate that LLM-powered evolutionary search is a promising paradigm for automated, interpretable hallucination detector design.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。