通过测试时提示提升闭源视觉语言模型性能，仅需一次调用即可生效。
Test-Time Hinting for Black-Box Vision-Language Models
- 训练轻量提示生成器，动态为输入添加针对性引导提示。
- 在多个闭源模型上实现准确率提升，且无需重新训练。
- 适用于无法访问权重的前沿闭源模型，通用性强。
测试时扩展（TTS）方法在大语言模型中表现优异，但在视觉语言模型（VLM）中的应用仍较少。现有VLM TTS方法大多需要开放模型权重或昂贵的重复采样，且主要在多模态数学与科学推理基准上评估，而非通用视觉理解任务。本文提出测试时提示（Test-Time Hinting），仅需一次VLM调用和黑盒API访问，即可显著提升闭源VLM性能。该方法基于观察：VLM错误常集中于重复出现的失败模式。因此，我们训练一个轻量提示生成器，针对给定测试输入预测应附加的“提示”，提供目标性上下文或流程指引，引导VLM避开其典型错误模式。实验表明，Test-Time Hinting在自然图像问答基准上提升了多个闭源VLM的准确率，且效果可泛化至未见过的基准和模型，无需重训提示生成器。
原文摘要 · Abstract (English)
Test-time scaling (TTS) methods have proven highly effective for LLMs, yet their application to vision-language models (VLMs) remains relatively underexplored. Existing VLM TTS methods largely require open-weight model access or expensive repeated sampling, and are evaluated primarily on multimodal mathematical and scientific reasoning benchmarks rather than general visual understanding tasks. In this paper, we propose Test-Time Hinting, a method that improves VLM performance via a single VLM call and requiring only black-box API access, which makes it broadly applicable to frontier closed-weight models. Our method is motivated by the observation that VLM errors tend to cluster around recurring failure patterns. We therefore train a lightweight hint generator model to predict, for a given test input, which "hint" should be prepended to the prompt, providing targeted contextual or procedural guidance that steers the VLM away from its characteristic failure modes. We show that Test-Time Hinting improves the accuracy of multiple closed-weight VLMs on natural-image VQA benchmarks and that these gains generalize to unseen benchmarks and VLMs without retraining the hint generator.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。