提出简单基线HapNet，应对边缘设备的不规则数据流挑战
Hedging Is Not All You Need: A Simple Baseline for Online Learning Under Haphazard Inputs
- 用自注意力替代复杂分叉结构，实现轻量可靠模型
- 在五个基准上表现媲美现有方法，支持可变窗口输入
- 适合实时边缘计算场景，无需在线反向传播
处理来自边缘设备等的不规则流数据极具挑战性，随时间推移输入数据会变得不一致，出现缺失、错误或新类型数据反复出现。现有方法依赖基于对冲的方案，需辅助丢弃、分叉结构和复杂网络设计。我们发现对冲可视为加权残差连接的特例，由此提出用普通自注意力近似。本文提出HapNet，一种可扩展、无需在线反向传播、适应多种输入类型的简单基线。现有方法仅适用于固定窗口，而我们引入更复杂的可变窗口场景——数据位置无关，现有方法无法解决。实验表明，该方法变体在该复杂场景下仍有效。在五个基准上广泛评估，性能具有竞争力。
原文摘要 · Abstract (English)
Handling haphazard streaming data, such as data from edge devices, presents a challenging problem. Over time, the incoming data becomes inconsistent, with missing, faulty, or new inputs reappearing. Therefore, it requires models that are reliable. Recent methods to solve this problem depend on a hedging-based solution and require specialized elements like auxiliary dropouts, forked architectures, and intricate network design. We observed that hedging can be reduced to a special case of weighted residual connection; this motivated us to approximate it with plain self-attention. In this work, we propose HapNet, a simple baseline that is scalable, does not require online backpropagation, and is adaptable to varying input types. All present methods are restricted to scaling with a fixed window; however, we introduce a more complex problem of scaling with a variable window where the data becomes positionally uncorrelated, and cannot be addressed by present methods. We demonstrate that a variant of the proposed approach can work even for this complex scenario. We extensively evaluated the proposed approach on five benchmarks and found competitive performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。