用相机参数当提示，让视觉语言模型自适应不同光照条件。
Measurement Plasticity: Sensor-Level Adaptation for Vision-Language Models
- 将相机曝光三要素作物理提示，推理时动态选最优拍摄视角。
- 在ImageNet-ES上比纯数字方法提升准确率，尤其在自动曝光场景。
- 无需反向传播或模型修改，适合部署于资源受限设备。
我们提出多视角物理提示（MVP）用于测试时自适应（TTA），这是一种前向传播框架，将TTA从令牌层面推进到光子层面，将相机曝光三角（即ISO、快门速度和光圈）视为物理提示。推理时，MVP基于源相似性得分获取选定的多个物理视图，对每个保留视图进行数字增强并筛选低熵预测结果，最后通过硬投票聚合预测。该选择-投票设计简单、校准友好，且无需梯度计算或模型修改。在ImageNet-ES和ImageNet-ES-Diverse上，MVP在自动曝光及与传统传感器控制结合的情况下均优于纯数字TTA。即使在减少参数候选以降低捕获延迟时，MVP仍保持有效性，展现出实际应用潜力。
原文摘要 · Abstract (English)
We propose Multi-View Physical-prompt (MVP) for Test-Time Adaptation (TTA), a forward-only framework that moves TTA from tokens to photons by treating the camera exposure triangle (i.e., ISO, shutter speed, and aperture) as physical prompts. At inference, MVP acquires selected multiple physical views using a source-affinity score, evaluates digitally augmented variants of each retained view and filters the lowest-entropy predictions, and aggregates predictions with hard voting. This selection-then-vote design is simple, calibration-friendly, and requires no gradients or model modifications. On ImageNet-ES and ImageNet-ES-Diverse, MVP outperforms digital-only TTA on both Auto-Exposure and a combination with conventional sensor control. MVP remains effective under reduced parameter candidates that lower capture latency, demonstrating its practicality.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。