arXiv:2606.02565cs.CV2026-06International Conf…

实时预测视觉任务需求,动态分配像素带宽聚焦关键区域。

Policy-based Foveated Imaging and Perception

论文配图:Policy-based Foveated Imaging and Perception
图 1 · 摘自论文原文
  • 基于传感器注意力策略,采集时动态分配像素资源。
  • 在严格像素预算下,任务性能显著优于现有方法。
  • 适用于高分辨率成像中带宽受限的实时感知场景。

超高清图像传感器能捕捉视觉感知任务所需的空间细节,但在实际带宽、延迟和功耗限制下,全分辨率采集与处理往往不可行。现有方法通过空间或时间降采样解决此问题,但会提前丢弃信息。本文提出一种实时、预测性且任务感知的中心化成像系统,直接在图像采集阶段运行。利用新兴的双流传感器架构,该方法在保持低分辨率全局上下文的同时,动态将有限的像素带宽分配给任务相关的兴趣区域。我们将中心化采集建模为传感器注意力策略学习问题,过去观测指导未来测量决策,实现感知-采集闭环。在多个感知任务的大量仿真中,本方法在严格像素预算下实现了高性能,并显著优于同带宽下的基线模型。我们在200兆像素双流传感器上进一步验证了系统,实测视频在真实带宽与延迟约束下完成,证明了任务驱动式采集时中心化成像的实际可行性。

原文摘要 · Abstract (English)

Ultra-high-resolution image sensors offer the potential to capture fine spatial details critical for many visual perception tasks, but acquiring and processing all pixels at full resolution is often infeasible under realistic bandwidth, latency, and power constraints. Existing approaches address this challenge through acquisition strategies such as spatial or temporal downsampling, which irrevocably discard information before task relevance can be assessed. In this work, we introduce a real-time, predictive, and task-aware foveated imaging system that operates directly at image acquisition time. Leveraging emerging dual-stream sensor architectures, our method dynamically allocates limited pixel bandwidth to task-relevant regions of interest while maintaining a low-resolution global context. We formulate foveated acquisition as a sensor attention policy-learning problem, in which past observations guide actions that determine future measurements, closing the perception-acquisition loop. Through extensive simulation across multiple perception tasks, we demonstrate that our approach achieves high task performance under strict pixel budgets and significantly outperforms relevant baselines operating at the same bandwidth. We further validate our system on a 200-megapixel dual-stream sensor, capturing real-world videos under realistic bandwidth and latency constraints, demonstrating the practical feasibility of task-driven, acquisition-time foveated imaging.

中心化成像传感器优化实时感知双流传感器

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。