arXiv:2410.01660cs.LGcs.AI2024-10ICLR被引 12

提出新方法提升生成模型安全性和采样效率,确保输出有统计保障。

Conformal Generative Modeling with Improved Sample Efficiency through Sequential Greedy Filtering

论文配图:Conformal Generative Modeling with Improved Sample Efficiency through Sequential Greedy Filtering
图 1 · 摘自论文原文
  • 通过迭代贪婪过滤逐步优化生成样本集
  • 显著减少专家人工验证次数,降低安全场景成本
  • 适用于需要可靠输出的自然语言与分子结构生成任务

生成模型缺乏严格的统计保证,难以应用于安全敏感场景。本文提出序列化生成模型共形预测方法 SCOPE-Gen,通过迭代贪心过滤从黑箱生成模型中采样并筛选,构建满足共形可接受性控制的预测集——即高概率包含至少一个有效样本。该方法利用生成过程的马尔可夫结构,使每一步的可接受性可独立控制。相比先前方法,大幅减少校准阶段的可接受性评估次数,特别适用于需专家手动验证的高成本场景。在自然语言生成和分子图扩展任务中的实验表明,该方法在保持较低预测集大小的同时,显著降低了评估开销。

原文摘要 · Abstract (English)

Generative models lack rigorous statistical guarantees for their outputs and are therefore unreliable in safety-critical applications. In this work, we propose Sequential Conformal Prediction for Generative Models (SCOPE-Gen), a sequential conformal prediction method producing prediction sets that satisfy a rigorous statistical guarantee called conformal admissibility control. This guarantee states that with high probability, the prediction sets contain at least one admissible (or valid) example. To this end, our method first samples an initial set of i.i.d. examples from a black box generative model. Then, this set is iteratively pruned via so-called greedy filters. As a consequence of the iterative generation procedure, admissibility of the final prediction set factorizes as a Markov chain. This factorization is crucial, because it allows to control each factor separately, using conformal prediction. In comparison to prior work, our method demonstrates a large reduction in the number of admissibility evaluations during calibration. This reduction is important in safety-critical applications, where these evaluations must be conducted manually by domain experts and are therefore costly and time consuming. We highlight the advantages of our method in terms of admissibility evaluations and cardinality of the prediction sets through experiments in natural language generation and molecular graph extension tasks.

生成模型共形预测安全生成采样效率

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。