用混合模型实现长文档高效摘要，不截断且速度快。
Efficient Extractive Summarization with MAMBA-Transformer Hybrids for Low-Resource Scenarios
- 结合Transformer语义与Mamba线性处理能力，不截断长文本。
- 在低资源下ROUGE-1最高提升0.23，全数据集显著优于基线。
- 适合数据少、文档长的场景，推理速度提升24%-27%。
长文档的抽取式摘要受限于二次复杂度，常需截断，限制了资源受限环境下的部署。本文提出首个用于抽取式摘要的Mamba-Transformer混合模型，融合预训练Transformer的语义能力与状态空间模型的线性时间处理特性。利用Mamba无需截断即可处理全文的能力，该方法在保持强摘要质量的同时保留上下文信息。架构包含：(1) 句子级语义的Transformer编码器，(2) 高效捕捉句间依赖的Mamba状态空间模型，(3) 句子相关性预测的线性分类器。在新闻、论辩和科学领域低资源条件下，本方法实现：(1) 显著优于BERTSUM和MATCHSUM，ArXiv上ROUGE-1提升+0.23，所有数据集均具有统计显著性（p < 0.001）；(2) 跨领域稳定表现，最长文档优势最明显；(3) 有限训练数据下仍具鲁棒性；(4) 新闻摘要任务推理速度提升24%-27%（CNN/DailyMail）。这是首个用于摘要的Transformer-状态空间混合架构，展示了在低资源场景下的显著性能提升。
原文摘要 · Abstract (English)
Extractive summarization of long documents is bottlenecked by quadratic complexity, often forcing truncation and limiting deployment in resource-constrained settings. We introduce the first Mamba-Transformer hybrid for extractive summarization, combining the semantic strength of pre-trained transformers with the linear-time processing of state space models. Leveraging Mamba's ability to process full documents without truncation, our approach preserves context while maintaining strong summarization quality. The architecture includes: (1) a transformer encoder for sentence-level semantics, (2) a Mamba state space model to capture inter-sentence dependencies efficiently, and (3) a linear classifier for sentence relevance prediction. Across news, argumentative, and scientific domains under low-resource conditions, our method achieves: (1) large gains over BERTSUM and MATCHSUM, including +0.23 ROUGE-1 on ArXiv and statistically significant improvements on all datasets (p < 0.001); (2) consistent advantages across domains, strongest on the longest documents; (3) robust performance with limited training data; and (4) 24-27% faster inference on news summarization (CNN/DailyMail). We introduce the first hybrid Transformer-state space architecture for summarization, showing significant ROUGE improvements in low-resource scenarios.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。