提出一种无需真实目标分布的条件生成样本质量评估方法。
Assessing Sample Quality in Conditional Generation under Compositional Shift
- 用训练数据分布计算每样本的可信度得分,结合真实感与属性忠实度。
- 在属性外推场景下,得分能有效区分生成质量并提升下游任务性能。
- 适用于预训练模型,可实时筛选或拒绝低质生成结果。
条件生成模型常用于控制生成,尤其在科学领域探索罕见、昂贵或未观测到的属性组合时极具价值。然而,在外推场景中,真实目标分布不可得,导致标准评价指标失效。本文提出一种后验的逐样本可信度评分机制,仅依赖训练数据分布,融合全局真实感(与真实数据流形的兼容性)和属性忠实度(与请求属性的接近程度),在观察属性覆盖充分的前提下,能有效比较外推生成结果。该评分可用于生成结果的过滤、排序与主动放弃,且可直接应用于现成预训练模型。在生物图像中,优选样本保留了更真实的形态结构,提升了下游预测性能;视觉基准测试也显示类似增益。此外,该评分可嵌入生成过程,实现生成前主动放弃。代码已开源。
原文摘要 · Abstract (English)
Conditional generators provide a natural tool for controllable generation, including settings where the desired condition is a new composition of observed attributes or experimental factors. In many applications, especially in scientific domains, such models are attractive to explore conditions for which real samples are rare, expensive, or not yet observed. However, this creates a circularity for evaluation: standard conditional quality metrics require a reference target distribution, but in the extrapolative regime that distribution is unavailable by definition. We address this problem with a post-hoc, per-sample trust score for assessing conditional samples using only the training distribution. The score combines two estimable quantities: global realism, measuring compatibility with the real data manifold, and attribute-wise faithfulness, measuring whether a sample is closer to the requested attributes than to plausible alternatives. We show that the score can recover meaningful comparisons across extrapolated generations, under a mild coverage condition on the observed attributes. These comparisons enable effective filtering, ranking, and abstention of generations and can be used directly on off-the-shelf pretrained models. In biological imaging, selected samples preserve real morphological structure better and improve downstream predictive performance, while similar gains are observed on controlled vision benchmarks. Finally, we show how the score can be applied during generation, enabling abstention before full decoding. Code is available at https://github.com/berkerdemirel/faithful-cond-gen.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。