用专家分工提升关键词提取效果，尤其适合小数据集。
SEKE: Specialised Experts for Keyword Extraction
- 采用专家混合模型，让不同专家专注处理文本不同部分。
- 在多个英文数据集上超越现有方法，小数据集表现更优。
- 能揭示专家分工模式，增强结果可解释性，适合研究者使用。
关键词提取旨在识别文档中最具描述性的词汇，实现海量异构文本的自动分类与摘要。鉴于真实场景中关键词识别需处理多样内容，本文提出基于专家混合（MoE）框架的监督式关键词提取方法SEKE。SEKE以DeBERTa为骨干模型，结合双向LSTM网络，利用可学习路由机制引导信息至专门化专家，使专家专注于输入空间的不同区域。该设计在训练数据较少的小型语料上仍能有效提取关键词。实验表明，不同数据规模与类型下，专家会分别聚焦标点、停用词、词性或命名实体等成分。方法在多个英文数据集上达到当前最优性能，并提升了模型可解释性。代码已开源。
原文摘要 · Abstract (English)
Keyword extraction involves identifying the most descriptive words in a document, allowing automatic categorisation and summarisation of large quantities of diverse textual data. Relying on the insight that real-world keyword detection often requires handling of diverse content, we propose a novel supervised keyword extraction approach based on the mixture of experts (MoE) technique. MoE uses a learnable routing sub-network to direct information to specialised experts, allowing them to specialise in distinct regions of the input space. SEKE, a mixture of Specialised Experts for supervised Keyword Extraction, uses DeBERTa as the backbone model and builds on the MoE framework, where experts attend to each token, by integrating it with a bidirectional Long short-term memory (BiLSTM) network, to allow successful extraction even on smaller corpora, where specialisation is harder due to lack of training data. The MoE framework also provides an insight into inner workings of individual experts, enhancing the explainability of the approach. We benchmark SEKE on multiple English datasets, achieving state-of-the-art performance compared to strong supervised and unsupervised baselines. Our analysis reveals that depending on data size and type, experts specialise in distinct syntactic and semantic components, such as punctuation, stopwords, parts-of-speech, or named entities. Code is available at https://github.com/matejMartinc/SEKE_keyword_extraction
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。