轻量级序列建模新架构，兼顾快速动态与长程依赖。
mGRADE: Minimal Recurrent Gating Meets Delay Convolutions for Lightweight Sequence Modeling
- 用可学习时序间隔卷积+轻量门控循环，实现跨时间尺度建模
- 内存占用减少8倍，长程选择性与快速动态建模性能仍保持领先
- 适合边缘设备部署，尤其适用于语音识别等低资源场景
多时间尺度序列建模需同时捕捉局部快速动态与全局慢速上下文，但在边缘设备严格的内存约束下仍具挑战。现有最先进的模型在固定内存开销下，常以牺牲长程选择性或高精度快速动态建模为代价。为此，我们提出mGRADE（最小门控循环架构与延迟嵌入），一种混合记忆系统：通过引入可学习时序间隔的卷积与轻量门控循环组件，结合跨时间尺度的归纳偏置。理论上证明，可学习间隔等价于延迟嵌入，能以参数高效方式重构部分观测的快速动态；门控循环组件则以极小内存开销选择性保留长程上下文。在长程竞技场（Long-Range Arena）及35类谷歌语音命令原始音频分类任务中，mGRADE相较其他先进模型内存占用减少达8倍，且性能保持竞争力。
原文摘要 · Abstract (English)
Multi-timescale sequence modeling relies on capturing both local fast dynamics and global slow context; yet, maintaining these capabilities under the strict memory constraints common to edge devices remains an open challenge. Current State-of-the-Art models with constant memory footprints trade off long-range selectivity and high-precision modeling of fast dynamics. To overcome this trade-off within a fixed memory budget, we propose mGRADE (minimally Gated Recurrent Architecture with Delay Embedding), a hybrid-memory system that introduces inductive biases across timescales by integrating a convolution with learnable temporal spacings with a lightweight gated recurrent component. We show theoretically that the learnable spacings are equivalent to a delay embedding, enabling parameter-efficient reconstruction of partially-observed fast dynamics, while the gated recurrent component selectively maintains long-range context with minimal memory overhead. On the challenging Long-Range Arena benchmark and 35-way Google Speech Commands raw audio classification task, mGRADE reduces the memory footprint by up to a factor of 8 compared to other State-of-the-Art models, while maintaining competitive performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。