arXiv:2512.06763cs.CV2025-12

联合优化相机硬件与动态控制算法,提升视觉任务表现。

JOCA: Task-Driven Joint Optimisation of Camera Hardware and Adaptive Camera Control Algorithms

  • 融合梯度与无导数优化,支持连续/离散参数及神经网络控制。
  • 在弱光和快速运动下,性能超越独立优化静态与动态参数的基线。
  • 适合需要高精度感知的智能相机系统设计者。

图像质量显著影响下游感知任务的表现。近期研究通过将相机系统与感知任务协同设计,提升了任务性能。然而,多数方法仅优化制造时固定的相机参数,而实际运行中需自适应调整的参数(如曝光设置)仍缺乏有效优化。本文提出一种联合优化相机硬件与自适应控制算法的方法,针对下游视觉任务进行优化。我们构建了一个统一优化框架,结合基于梯度与无导数优化方法,可处理连续与离散参数、不可微成像过程以及基于神经网络的自适应控制算法。为应对运动模糊等不可微效应,提出DF-Grad,即利用无导数优化器信号训练自适应控制网络,并结合无监督任务驱动学习。实验表明,该方法在弱光与快速运动等挑战性条件下,优于分别优化静态与动态参数的基线。结果证明,联合优化硬件参数与自适应控制算法能显著提升感知性能,并提供统一的任务驱动相机系统设计范式。

原文摘要 · Abstract (English)

The quality of captured images strongly influences the performance of downstream perception tasks. Recent works on co-designing camera systems with perception tasks have shown improved task performance. However, most prior approaches focus on optimising fixed camera parameters set at manufacturing, while many parameters, such as exposure settings, require adaptive control at runtime. This paper introduces a method that jointly optimises camera hardware and adaptive camera control algorithms with downstream vision tasks. We present a unified optimisation framework that integrates gradient-based and derivative-free methods, enabling support for both continuous and discrete parameters, non-differentiable image formation processes, and neural network-based adaptive control algorithms. To address non-differentiable effects such as motion blur, we propose DF-Grad, a hybrid optimisation strategy that trains adaptive control networks using signals from a derivative-free optimiser alongside unsupervised task-driven learning. Experiments show that our method outperforms baselines that optimise static and dynamic parameters separately, particularly under challenging conditions such as low light and fast motion. These results demonstrate that jointly optimising hardware parameters and adaptive control algorithms improves perception performance and provides a unified approach to task-driven camera system design.

相机系统协同设计自适应控制感知优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。