用大模型生成相关性先验,自动判断数据发现是否值得深入研究。
Exploiting LLMs for Automatic Hypothesis Assessment via a Logit-Based Calibrated Prior
- 基于大模型输出的对数转换,构建可校准的相关性先验分布。
- 在2096个真实变量对上预测皮尔逊相关系数,准确率达78.8%。
- 能识别未见过的相关性模式,适合数据探索与科研选题推荐。
随着假设生成日益自动化,新的瓶颈出现:假设评估困难。现代系统可挖掘出数千个统计关系(如相关性、趋势、因果链),却难以判断哪些具有新颖性或值得专家关注。本文研究假设生成的互补问题:如何自动评估大量统计关系中哪些值得进一步探索?聚焦于相关性,因其是探索性数据分析的常见起点,常作为形成更深层科学或因果假设的基础。我们提出利用大模型权重中编码的海量知识,构建变量对相关性的先验分布。若大模型预期的值与实际观察值一致,则该相关性不意外;反之则可能新颖。提出基于对数的校准先验(Logit-based Calibrated Prior),将模型原始输出对数转换为连续的预测分布。在包含2,096个真实世界变量对的基准上测试,该方法预测皮尔逊相关系数的符号准确率达78.8%,平均绝对误差为0.26,95%可信区间覆盖率为89.2%。优于微调的RoBERTa分类器在二分类任务上的表现,并在假设排序中获得更高precision@K。此外,该先验可泛化至预训练阶段未见的相关性,体现上下文敏感推理而非记忆。
原文摘要 · Abstract (English)
As hypothesis generation becomes increasingly automated, a new bottleneck has emerged: hypothesis assessment. Modern systems can surface thousands of statistical relationships-correlations, trends, causal links-but offer little guidance on which ones are novel, non-trivial, or worthy of expert attention. In this work, we study the complementary problem to hypothesis generation: automatic hypothesis assessment. Specifically, we ask: given a large set of statistical relationships, can we automatically assess which ones are novel and worth further exploration? We focus on correlations as they are a common entry point in exploratory data analysis that often serve as the basis for forming deeper scientific or causal hypotheses. To support automatic assessment, we propose to leverage the vast knowledge encoded in LLMs' weights to derive a prior distribution over the correlation value of a variable pair. If an LLM's prior expects the correlation value observed, then such correlation is not surprising, and vice versa. We propose the Logit-based Calibrated Prior, an LLM-elicited correlation prior that transforms the model's raw output logits into a calibrated, continuous predictive distribution over correlation values. We evaluate the prior on a benchmark of 2,096 real-world variable pairs and it achieves a sign accuracy of 78.8%, a mean absolute error of 0.26, and 95% credible interval coverage of 89.2% in predicting Pearson correlation coefficient. It also outperforms a fine-tuned RoBERTa classifier in binary correlation prediction and achieves higher precision@K in hypothesis ranking. We further show that the prior generalizes to correlations not seen during LLM pretraining, reflecting context-sensitive reasoning rather than memorization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。