提出新型稀疏注意力框架，高效处理长文本摘要。
BiSparse-AAS: Bilinear Sparse Attention and Adaptive Spans Framework for Scalable and Efficient Text Summarization
- 结合稀疏注意力与自适应跨度，动态聚焦关键信息。
- 在CNN/DailyMail上提升ROUGE达68.1%，XSum上52.6%。
- 适合大规模、长文本的实时摘要应用。
基于Transformer的文本摘要模型虽性能优异，但其二次复杂度限制了长文档的可扩展性。本文提出BiSparse-AAS（双线性稀疏注意力与自适应跨度框架），融合稀疏注意力、自适应跨度和双线性注意力，解决上述问题。稀疏注意力通过聚焦输入中最具相关性的部分降低计算开销，自适应跨度动态调整关注范围，双线性注意力则在优化后的上下文中建模复杂词元交互。该框架在抽取式与抽象式摘要任务中均显著优于现有基线，在CNN/DailyMail数据集上平均ROUGE提升约68.1%，在XSum上提升52.6%，同时在OpenWebText和Gigaword数据集上保持优异表现。该方法兼顾效率、可扩展性与长序列建模能力，为实际文本摘要应用提供统一可行方案。
原文摘要 · Abstract (English)
Transformer-based architectures have advanced text summarization, yet their quadratic complexity limits scalability on long documents. This paper introduces BiSparse-AAS (Bilinear Sparse Attention with Adaptive Spans), a novel framework that combines sparse attention, adaptive spans, and bilinear attention to address these limitations. Sparse attention reduces computational costs by focusing on the most relevant parts of the input, while adaptive spans dynamically adjust the attention ranges. Bilinear attention complements both by modeling complex token interactions within this refined context. BiSparse-AAS consistently outperforms state-of-the-art baselines in both extractive and abstractive summarization tasks, achieving average ROUGE improvements of about 68.1% on CNN/DailyMail and 52.6% on XSum, while maintaining strong performance on OpenWebText and Gigaword datasets. By addressing efficiency, scalability, and long-sequence modeling, BiSparse-AAS provides a unified, practical solution for real-world text summarization applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。