根据问题自动选最划算的图像清晰度，省成本又保准确。
VOILA: Value-of-Information Guided Fidelity Selection for Cost-Aware Multimodal Question Answering
- 先估问题对应不同画质的答对概率，再选性价比最高的画质
- 成本降低50%-60%，准确率仍保持在90%-95%之间
- 适合资源受限场景，尤其适用于大模型多模态问答
尽管高分辨率视觉输入的获取与处理代价高昂，多数多模态视觉语言系统仍采用固定画质。本文提出VOILA框架，通过信息价值驱动实现视觉问答中自适应画质选择，优化模型执行前的信息获取策略。给定问题后，系统采用两阶段流程：基于问题特征，梯度提升回归器预测各画质下的答对可能性；随后使用等序校准器校准概率以支持可靠决策。系统选择在预估准确率与检索成本权衡下预期收益最大的最低成本画质。我们在五个数据集（VQA-v2, GQA, TextVQA, LoCoMo, FloodNet）和六种参数量为7B-235B的视觉语言模型上评估了该方法，覆盖三种部署场景。结果表明，VOILA在各类查询类型和模型架构下均实现50%-60%的成本节约，同时保留90%-95%的全分辨率准确率，证明了预检索画质选择对资源受限环境下多模态推理优化至关重要。
原文摘要 · Abstract (English)
Despite significant costs from retrieving and processing high-fidelity visual inputs, most multimodal vision-language systems operate at fixed fidelity levels. We introduce VOILA, a framework for Value-Of-Information-driven adaptive fidelity selection in Visual Question Answering (VQA) that optimizes what information to retrieve before model execution. Given a query, VOILA uses a two-stage pipeline: a gradient-boosted regressor estimates correctness likelihood at each fidelity from question features alone, then an isotonic calibrator refines these probabilities for reliable decision-making. The system selects the minimum-cost fidelity maximizing expected utility given predicted accuracy and retrieval costs. We evaluate VOILA across three deployment scenarios using five datasets (VQA-v2, GQA, TextVQA, LoCoMo, FloodNet) and six Vision-Language Models (VLMs) with 7B-235B parameters. VOILA consistently achieves 50-60% cost reductions while retaining 90-95% of full-resolution accuracy across diverse query types and model architectures, demonstrating that pre-retrieval fidelity selection is vital to optimize multimodal inference under resource constraints.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。