用专家分工提升定义生成质量，性能比之前最好方法高7%。
LM-Lexicon: Improving Definition Modeling via Harmonizing Semantic Experts
- 将定义任务拆成专业领域，小模型分别做专家
- 聚类策略让定义质量提升近10%，路由机制效率增1%
- 适合需要高质量语义生成的场景，如知识库构建
我们提出LM-Lexicon，一种通过数据聚类、语义专家学习和稀疏专家混合架构融合的定义建模方法。该方法将定义任务分解为特定语义领域，训练小型语言模型作为领域专家，显著提升性能：在五个主流基准上相比先前最优模型，BLEU得分提升7%。实证表明：1）聚类策略实现细粒度专家专精，定义质量提升近10%；2）基于语义的域级路由机制相比传统词元级路由，专家利用率提高1%；3）通过测试时计算资源扩展与语义专家规模增加可进一步提升性能。本工作推动定义建模发展，也为语义密集型应用中的高效语言模型设计提供启示。
原文摘要 · Abstract (English)
We introduce LM-Lexicon, an innovative definition modeling approach that incorporates data clustering, semantic expert learning, and model merging using a sparse mixture-of-experts architecture. By decomposing the definition modeling task into specialized semantic domains, where small language models are trained as domain experts, LM-Lexicon achieves substantial improvements (+7% BLEU score compared with the prior state-of-the-art model) over existing methods on five widely used benchmarks. Empirically, we demonstrate that 1) the clustering strategy enables fine-grained expert specialization with nearly 10% improvement in definition quality; 2) the semantic-aware domain-level routing mechanism achieves higher expert efficacy (+1%) than conventional token-level routing; and 3) further performance gains can be obtained through test-time compute and semantic expert scaling. Our work advances definition modeling while providing insights into the development of efficient language models for semantic-intensive applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。