让相机根据模型需求自动调优,提升视觉识别准确率。
Adaptive Camera Sensor for Vision Models
- 基于模型视角动态调整传感器参数,实时优化图像质量。
- 在多个数据集上显著提升模型精度,且捕获延迟低。
- 无需额外训练,适合部署在资源受限的视觉系统中。
领域偏移仍是深度学习视觉任务中的持续挑战,通常需大量模型修改或标注数据来应对。受人类视觉感知启发——通过调节镜头而非过度训练大脑来适应环境,我们提出Lens:一种新型相机传感器控制方法,通过从模型视角捕捉高质量图像,而非依赖传统以人为主的传感器设置。Lens轻量且能实时适配特定模型与场景。其核心是VisiT,一种无需训练、仅依赖置信度评分的模型专属质量评估指标,可在测试时对未标注样本进行无成本评估。为验证Lens,我们引入ImageNet-ES Diverse,一个涵盖自然传感器与光照扰动的新基准数据集。在ImageNet-ES和ImageNet-ES Diverse上的大量实验表明,Lens显著提升了多种传感器控制与模型改进方案的准确性,同时保持低延迟。Lens有效缓解了大模型间的性能差异,并可与模型优化技术协同增效。代码与数据集已开源于github.com/Edw2n/Lens.git。
原文摘要 · Abstract (English)
Domain shift remains a persistent challenge in deep-learning-based computer vision, often requiring extensive model modifications or large labeled datasets to address. Inspired by human visual perception, which adjusts input quality through corrective lenses rather than over-training the brain, we propose Lens, a novel camera sensor control method that enhances model performance by capturing high-quality images from the model's perspective rather than relying on traditional human-centric sensor control. Lens is lightweight and adapts sensor parameters to specific models and scenes in real-time. At its core, Lens utilizes VisiT, a training-free, model-specific quality indicator that evaluates individual unlabeled samples at test time using confidence scores without additional adaptation costs. To validate Lens, we introduce ImageNet-ES Diverse, a new benchmark dataset capturing natural perturbations from varying sensor and lighting conditions. Extensive experiments on both ImageNet-ES and our new ImageNet-ES Diverse show that Lens significantly improves model accuracy across various baseline schemes for sensor control and model modification while maintaining low latency in image captures. Lens effectively compensates for large model size differences and integrates synergistically with model improvement techniques. Our code and dataset are available at github.com/Edw2n/Lens.git.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。