无需标签或概率，用几何信号控制大模型生成不确定性。
Unsupervised Conformal Inference: Bootstrapping and Alignment to Control LLM Uncertainty
- 用响应嵌入的格拉姆矩阵构造异常分数，实现无监督检测。
- 在多个数据集上覆盖率达标且阈值更紧致稳定，减少幻觉现象。
- 适合需实时过滤输出的部署场景，兼容API且无需人工标注。
部署黑箱大模型时，缺乏词级概率或真实标签，难以管理不确定性。本文提出一种面向生成任务的无监督自校准推断框架，融合：(i) 基于响应嵌入格拉姆矩阵的适配大模型异常分数；(ii) 结合自助法的无条件置信区间（BB-UCP），通过聚合残差提升分位数精度，同时保证分布无关、有限样本覆盖性；(iii) 自校准对齐机制，仅调节一个严格度参数τ，使用户指定的判据（如事实性提升）在未见批次中以≥1−α的概率成立。在多个基准数据集上，该方法实现接近名义覆盖率，阈值更紧致、更稳定，显著降低幻觉程度，优于计算开销相近的轻量级逐条检测器。最终形成一个无需标签、兼容API的测试时过滤门，将几何信号转化为可校准、目标对齐的决策。
原文摘要 · Abstract (English)
Deploying black-box LLMs requires managing uncertainty in the absence of token-level probability or true labels. We propose introducing an unsupervised conformal inference framework for generation, which integrates: generative models, incorporating: (i) an LLM-compatible atypical score derived from response-embedding Gram matrix, (ii) UCP combined with a bootstrapping variant (BB-UCP) that aggregates residuals to refine quantile precision while maintaining distribution-free, finite-sample coverage, and (iii) conformal alignment, which calibrates a single strictness parameter $τ$ so a user predicate (e.g., factuality lift) holds on unseen batches with probability $\ge 1-α$. Across different benchmark datasets, our gates achieve close-to-nominal coverage and provide tighter, more stable thresholds than split UCP, while consistently reducing the severity of hallucination, outperforming lightweight per-response detectors with similar computational demands. The result is a label-free, API-compatible gate for test-time filtering that turns geometric signals into calibrated, goal-aligned decisions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。