arXiv:2608.16523cs.CV2026-08

用令牌化方法直接处理事件流,提升事件相机强化学习性能

FLEET: Token-Based Feature Extraction for Event Camera-based Reinforcement Learning

论文配图:FLEET: Token-Based Feature Extraction for Event Camera-based Reinforcement Learning
图 1 · 摘自论文原文
  • 将事件流直接转为固定长度的令牌表示,避免传统网格聚合
  • 在新基准上超越现有最先进方法,对观测频率变化更鲁棒
  • 无需预训练数据,支持端到端学习,适合高帧率实时控制

事件相机产生异步、高频的数据流,具有空间稀疏性和低延迟特性,理论上非常适合控制策略设计。然而,现有强化学习方法未能充分利用这些优势:基于CNN的方法将事件聚合为稀疏网格,导致计算成本随传感器分辨率上升,并模糊时间信息;现有生成式基线依赖轨迹数据进行预训练。本文提出FLEET(基于高效令牌化的事件特征提取),直接处理事件序列,利用随机傅里叶特征和交叉注意力机制,将可变长度事件流压缩为固定大小的潜在表示。该架构使特征提取器的推理成本与传感器分辨率解耦,实现无需辅助损失的端到端学习。我们在一个新型高吞吐量基准上验证了FLEET,结果表明其序列化方法优于当前最优性能,并对观测频率变化表现出更强鲁棒性。

原文摘要 · Abstract (English)

Event cameras generate asynchronous, high-frequency data streams offering spatially sparse information at lower latency than traditional cameras. In principle, these properties should be ideal for the design of control policies. However, reinforcement learning research in this field remains limited as existing approaches fail to fully exploit the sensor's properties. CNN-based methods negate the sensors benefits by aggregating events into sparse grids. This couples compute cost to sensor resolution and blurs the temporal information. Meanwhile, existing generative baselines rely on the availability of trajectory data to pretrain the model. We propose FLEET (Feature Learning from Events via Efficient Tokenization), a feature extractor that processes event sequences directly. Leveraging random Fourier features and cross-attention, our architecture compresses variable streams into fixed-size latent representations. This decouples inference cost of the feature extractor's backbone from the sensor's resolution, enabling end-to-end learning without auxiliary losses. We validate FLEET on a new, high-throughput benchmark. The results demonstrate that our sequence-based approach surpasses SOTA performance and exhibits superior robustness to variations in observation frequencies.

事件相机强化学习特征提取令牌化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。