提出TRON框架,让多模态大模型生成有风险控制的可信回答。
Sample then Identify: A General Framework for Risk Control and Assessment in Multimodal Large Language Models
- 先采样再筛选:用新置信度分数生成最小响应集。
- 误差率受控在用户指定风险水平内,跨8个模型4个数据集有效。
- 首次分析开放场景语义冗余,提供更高效的评估指标。
多模态大语言模型(MLLM)在各类任务中展现出巨大潜力,但仍面临信任度挑战。现有研究虽采用分割合取预测(SCP)为语言模型构建具有统计保障的预测集,但通常依赖模型内部logits或局限于多选场景,难以适应动态开放环境。本文提出TRON,一种适用于支持采样的任意MLLM的两步风险控制与评估通用框架。该框架包含两个核心组件:(1) 新型合取得分,用于生成最小规模的响应集;(2) 非合取得分,基于自洽性理论识别高质量响应,通过设定两个特定风险水平控制错误率。此外,我们首次在开放场景中研究预测集内的语义冗余,提出基于平均集合大小的新评估指标。在四个视频问答(VideoQA)数据集上,使用八种MLLM的全面实验表明,TRON能将误差率稳定控制在用户指定的风险水平之内。同时,去重后的预测集在不同风险水平下兼具适应性、效率与稳定性。
原文摘要 · Abstract (English)
Multimodal Large Language Models (MLLMs) exhibit promising advancements across various tasks, yet they still encounter significant trustworthiness issues. Prior studies apply Split Conformal Prediction (SCP) in language modeling to construct prediction sets with statistical guarantees. However, these methods typically rely on internal model logits or are restricted to multiple-choice settings, which hampers their generalizability and adaptability in dynamic, open-ended environments. In this paper, we introduce TRON, a two-step framework for risk control and assessment, applicable to any MLLM that supports sampling in both open-ended and closed-ended scenarios. TRON comprises two main components: (1) a novel conformal score to sample response sets of minimum size, and (2) a nonconformity score to identify high-quality responses based on self-consistency theory, controlling the error rates by two specific risk levels. Furthermore, we investigate semantic redundancy in prediction sets within open-ended contexts for the first time, leading to a promising evaluation metric for MLLMs based on average set size. Our comprehensive experiments across four Video Question-Answering (VideoQA) datasets utilizing eight MLLMs show that TRON achieves desired error rates bounded by two user-specified risk levels. Additionally, deduplicated prediction sets maintain adaptiveness while being more efficient and stable for risk assessment under different risk levels.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。