用句子嵌入空间变换实现轻量级文本简化，效果媲美大模型。
Text Simplification with Sentence Embeddings
- 在句子嵌入空间中学习高低复杂度文本的转换映射。
- 小规模网络在简化任务上超越序列到序列模型和大模型。
- 可跨语言、跨数据集应用，适合资源受限场景。
句子嵌入可被解码以近似生成它们的原始文本。本文在文本简化任务中探索这一现象，发现重建的句子嵌入能保留原文复杂度特征。我们采用小型前馈神经网络，有效学习高复杂度与低复杂度文本嵌入之间的转换关系。在与序列到序列模型及大语言模型方法的对比中，展示了在更小学习设置下的良好表现。此外，该转换方法成功应用于未见过的简化数据集（MedEASI）以及训练数据之外的语言（西班牙语、德语）。研究结论表明，在句子嵌入空间中学习转换是未来自然语言生成任务的重要方向，有望实现小型但高效的文本简化模型。
原文摘要 · Abstract (English)
Sentence embeddings can be decoded to give approximations of the original texts used to create them. We explore this effect in the context of text simplification, demonstrating that reconstructed text embeddings preserve complexity levels. We experiment with a small feed forward neural network to effectively learn a transformation between sentence embeddings representing high-complexity and low-complexity texts. We provide comparison to a Seq2Seq and LLM-based approach, showing encouraging results in our much smaller learning setting. Finally, we demonstrate the applicability of our transformation to an unseen simplification dataset (MedEASI), as well as datasets from languages outside the training data (ES,DE). We conclude that learning transformations in sentence embedding space is a promising direction for future research and has potential to unlock the ability to develop small, but powerful models for text simplification and other natural language generation tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。