将材料知识融入分词，提升科学文本处理精度
Incorporating Domain Knowledge into Materials Tokenization
- 基于材料知识库训练模型，智能识别并保护材料概念完整性
- 在生成和分类任务中分别提升4%和2%性能
- 适合材料科学与领域知识融合的NLP研究者
尽管语言模型在材料科学中应用日益广泛，但传统基于频率的分词方法源自自然语言处理，常导致过度碎片化和语义丢失，无法保持材料概念的结构与语义完整。为此，我们提出MATTE，一种将材料知识融入分词的新方法。该方法基于在材料知识库上训练的MatDetector，结合优先合并材料概念的重排序机制，在分词过程中保留已识别材料概念的结构完整性，避免碎片化，确保语义不丢失。实验表明，MATTE在生成和分类任务中分别实现平均4%和2%的性能提升。结果强调了领域知识在科学文本分词策略中的关键作用。代码开源：https://github.com/yerimoh/MATTER
原文摘要 · Abstract (English)
While language models are increasingly utilized in materials science, typical models rely on frequency-centric tokenization methods originally developed for natural language processing. However, these methods frequently produce excessive fragmentation and semantic loss, failing to maintain the structural and semantic integrity of material concepts. To address this issue, we propose MATTER, a novel tokenization approach that integrates material knowledge into tokenization. Based on MatDetector trained on our materials knowledge base and a re-ranking method prioritizing material concepts in token merging, MATTER maintains the structural integrity of identified material concepts and prevents fragmentation during tokenization, ensuring their semantic meaning remains intact. The experimental results demonstrate that MATTER outperforms existing tokenization methods, achieving an average performance gain of $4\%$ and $2\%$ in the generation and classification tasks, respectively. These results underscore the importance of domain knowledge for tokenization strategies in scientific text processing. Our code is available at https://github.com/yerimoh/MATTER
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。