用视觉语言模型迭代评估,实现跨相机的稳定白平衡。
White-Balance First, Adjust Later: Cross-Camera Color Constancy via Vision-Language Evaluation

- 以视觉语言模型反馈引导白平衡迭代修正
- 跨相机测试下比现有方法提升5.2%~11.3%均值性能
- 适合需要高鲁棒性色彩还原的多相机系统
颜色恒常性旨在保持物体在不同光照下的颜色一致。基于学习的模型在跨相机泛化上表现不佳,因其易过拟合于训练相机的颜色响应特性。我们提出VLM-CC,一种基于反馈的迭代优化框架,将颜色恒常性重构为感知反馈过程。不直接从原始输入估计光源,而是通过当前估计进行白平衡处理,并转换为伪sRGB图像。轻量级LoRA微调的视觉语言模型(VLM)评估该图像,识别主要残留色偏并提供定性反馈。反馈被映射为残余光照方向(红/绿/蓝),用于更新光源估计直至收敛。核心思想是将颜色恒常性视为基于视觉语言模型评估的迭代感知反馈问题,替代传统的直接RGB回归。通过用感知反馈替代直接估计,VLM-CC在多个数据集上实现了跨相机颜色恒常性的最先进鲁棒性。代码将公开于https://github.com/NothingIknow/VLM-CC。
原文摘要 · Abstract (English)
Color constancy aims to keep object colors consistent under varying illumination. Cross-camera generalization in color constancy remains challenging because learning-based models often overfit to the color response characteristics of the training camera, resulting in degraded performance on images captured by other cameras. We propose VLM-CC, a feedback-guided framework that formulates color constancy as an iterative refinement process. Instead of directly estimating the illuminant from raw input, VLM-CC performs iterative correction driven by vision-language model (VLM)-based evaluation. At each iteration, the image is white-balanced using the current estimate and converted to pseudo-sRGB. A lightweight LoRA-tuned VLM then assesses the corrected image, identifying the dominant residual color cast and providing qualitative feedback. This feedback is mapped to a residual illumination direction (red, green, or blue) and used to update the illuminant estimate until convergence. Our key idea is to reframe color constancy as an iterative perceptual feedback problem, leveraging VLM evaluation instead of direct RGB regression. By replacing direct RGB estimation with VLM-guided perceptual feedback, VLM-CC achieves state-of-the-art robustness in cross-camera color constancy across multiple datasets. Code will be available at https://github.com/NothingIknow/VLM-CC.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。