用可学习的图小波变换替代传统注意力，提升结构化序列建模效率
Graph Laplacian Wavelet Transformer via Learnable Spectral Decomposition
- 基于显式图拉普拉斯构造多尺度小波变换
- 计算与内存复杂度从二次降至线性
- 适合语法/语义结构化的自然语言任务
现有针对结构化语言任务的序列到序列模型严重依赖点积自注意力机制，导致计算和内存复杂度随输入长度N呈二次增长。本文提出图小波变压器（GWT），用基于句法或语义解析生成的显式图拉普拉斯定义的可学习多尺度小波变换，替代这一瓶颈。分析表明，多尺度谱分解为图结构序列建模提供了可解释、高效且富有表现力的替代方案。
原文摘要 · Abstract (English)
Existing sequence to sequence models for structured language tasks rely heavily on the dot product self attention mechanism, which incurs quadratic complexity in both computation and memory for input length N. We introduce the Graph Wavelet Transformer (GWT), a novel architecture that replaces this bottleneck with a learnable, multi scale wavelet transform defined over an explicit graph Laplacian derived from syntactic or semantic parses. Our analysis shows that multi scale spectral decomposition offers an interpretable, efficient, and expressive alternative to quadratic self attention for graph structured sequence modeling.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。