用视觉Transformer评估模型输出质量，准确率超95%
IO Transformer: Evaluating SwinV2-Based Reward Models for Computer Vision
- 基于SwinV2构建输入输出评估模型，判断生成结果好坏
- 在纯输入依赖任务中达100%准确率，复杂任务仍超95.4%
- 适合需自动评估视觉模型输出质量的研究者
Transformer及其变体在文本、视觉和语音识别任务中已达到顶尖性能，但针对评估其他模型输出质量的Transformer研究较少。本文提出基于SwinV2的输入-输出变压器（IO Transformer）与仅输出变压器，用于推理质量评估、数据分类和策略优化。实验表明，在输出完全依赖输入的任务中，IO Transformer在Change Dataset 25（CD25）上实现完美评估准确率；在输出不完全依赖输入的场景中，改进的Swin V2架构在IO Segmentation Dataset上取得95.41%的最高得分，优于IO Transformer。本工作拓展了Transformer在计算机视觉奖励建模中的应用，并为优化此类模型提供关键见解。
原文摘要 · Abstract (English)
Transformers and their derivatives have achieved state-of-the-art performance across text, vision, and speech recognition tasks. However, minimal effort has been made to train transformers capable of evaluating the output quality of other models. This paper examines SwinV2-based reward models, called the Input-Output Transformer (IO Transformer) and the Output Transformer. These reward models can be leveraged for tasks such as inference quality evaluation, data categorization, and policy optimization. Our experiments demonstrate highly accurate model output quality assessment across domains where the output is entirely dependent on the input, with the IO Transformer achieving perfect evaluation accuracy on the Change Dataset 25 (CD25). We also explore modified Swin V2 architectures. Ultimately Swin V2 remains on top with a score of 95.41 % on the IO Segmentation Dataset, outperforming the IO Transformer in scenarios where the output is not entirely dependent on the input. Our work expands the application of transformer architectures to reward modeling in computer vision and provides critical insights into optimizing these models for various tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。