用Transformer搜索神经网络结构，提升翻译效果。
A Transformer-based Neural Architecture Search Method
- 基于Transformer设计跨多头注意力的搜索方法。
- 引入困惑度辅助评估，搜索结果优于仅用BLEU的基线模型。
- 适合需要优化翻译质量的研究者或工程师。
本文提出一种基于Transformer架构的神经架构搜索方法，通过搜索不同编码器与解码器组合下的多头注意力计算方式，以寻找具备更优翻译性能的神经网络结构。除使用BLEU分数外，还引入困惑度作为辅助评估指标，并利用多目标遗传算法迭代优化种群中的每个个体。实验结果表明，该算法搜索出的网络结构在各项指标上均优于所有基线模型；且引入困惑度作为辅助评估指标，能发现比仅依赖BLEU分数更优的模型。
原文摘要 · Abstract (English)
This paper presents a neural architecture search method based on Transformer architecture, searching cross multihead attention computation ways for different number of encoder and decoder combinations. In order to search for neural network structures with better translation results, we considered perplexity as an auxiliary evaluation metric for the algorithm in addition to BLEU scores and iteratively improved each individual neural network within the population by a multi-objective genetic algorithm. Experimental results show that the neural network structures searched by the algorithm outperform all the baseline models, and that the introduction of the auxiliary evaluation metric can find better models than considering only the BLEU score as an evaluation metric.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。