用多模态嵌入提升摘要事实一致性，支持跨语言生成。
Using Multimodal and Language-Agnostic Sentence Embeddings for Abstractive Summarization
- 引入多语言句子嵌入与实体注入机制增强事实准确
- 在低资源语言上表现优于基线，摘要更简洁抽象
- 兼容文本与语音输入，适合跨语言摘要任务
抽象式摘要通过生成新句子实现灵活重述，但易产生幻觉。本文利用LaBSE、SONAR、BGE-M3等预训练模型提取的多模态多语言句子嵌入，结合改进的BART-based法语模型，提出SBARThez框架。引入命名实体注入机制，在解码器输入中添加分词后的实体，提升生成摘要的事实一致性。该框架支持文本与语音输入，具备跨语言摘要能力，在低资源语言上性能优于词粒度基线，生成摘要更简洁且更具抽象性。
原文摘要 · Abstract (English)
Abstractive summarization aims to generate concise summaries by creating new sentences, allowing for flexible rephrasing. However, this approach can be vulnerable to inaccuracies, particularly `hallucinations' where the model introduces non-existent information. In this paper, we leverage the use of multimodal and multilingual sentence embeddings derived from pretrained models such as LaBSE, SONAR, and BGE-M3, and feed them into a modified BART-based French model. A Named Entity Injection mechanism that appends tokenized named entities to the decoder input is introduced, in order to improve the factual consistency of the generated summary. Our novel framework, SBARThez, is applicable to both text and speech inputs and supports cross-lingual summarization; it shows competitive performance relative to token-level baselines, especially for low-resource languages, while generating more concise and abstract summaries.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。