用双向LSTM+注意力机制,小模型实现高效高质翻译。
Efficient Machine Translation with a BiLSTM-Attention Approach
- 用双向LSTM编码+注意力解码,提升上下文理解能力。
- 在WMT14数据集上超越主流Transformer,模型体积更小。
- 适合移动端、边缘设备等资源受限场景使用。
随着自然语言处理技术的快速发展,机器翻译的准确率与效率成为研究热点。本文提出一种新型序列到序列(Seq2Seq)模型,在提升翻译质量的同时显著降低模型存储空间需求。该模型采用双向长短期记忆网络(Bi-LSTM)作为编码器,捕捉输入序列的上下文信息;解码器引入注意力机制,增强对关键信息的关注能力。相比当前主流的Transformer模型,本模型在WMT14机器翻译数据集上表现更优,且模型规模更小。研究首先阐述模型架构的设计原则与创新点,随后通过一系列实验验证其有效性,包括不同语对的性能评估及与传统Seq2Seq模型的对比分析。结果表明,该模型在保持翻译准确率的同时,大幅降低存储需求,对资源受限场景下的翻译应用具有重要意义。代码已开源:https://github.com/mindspore-lab/models/tree/master/research/arxiv_papers/miniformer。感谢MindSpore社区的支持。
原文摘要 · Abstract (English)
With the rapid development of Natural Language Processing (NLP) technology, the accuracy and efficiency of machine translation have become hot topics of research. This paper proposes a novel Seq2Seq model aimed at improving translation quality while reducing the storage space required by the model. The model employs a Bidirectional Long Short-Term Memory network (Bi-LSTM) as the encoder to capture the context information of the input sequence; the decoder incorporates an attention mechanism, enhancing the model's ability to focus on key information during the translation process. Compared to the current mainstream Transformer model, our model achieves superior performance on the WMT14 machine translation dataset while maintaining a smaller size. The study first introduces the design principles and innovative points of the model architecture, followed by a series of experiments to verify the effectiveness of the model. The experimental includes an assessment of the model's performance on different language pairs, as well as comparative analysis with traditional Seq2Seq models. The results show that while maintaining translation accuracy, our model significantly reduces the storage requirements, which is of great significance for translation applications in resource-constrained scenarios. our code are available at https://github.com/mindspore-lab/models/tree/master/research/arxiv_papers/miniformer. Thanks for the support provided by MindSpore Community.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。