给BERT注入汉字结构信息，提升生僻字识别能力。
CNM-BERT: A Drop-In Structural Embedding for Chinese Characters via Ideographic Description Sequences
- 将汉字拆解为构形序列，用树状MLP编码结构特征
- 在长尾和未登录字上结构准确率提升9.8，部件F1提升7.7
- 无需改动原模型，适配各类中文自然语言任务
基于标记的编码器如BERT将汉字视为原子符号，忽略了其递归的书写结构，导致模型依赖上下文共现，对罕见及未登录字表现不佳。我们提出组合网络模型（CNM），一种轻量级增强方法，通过将汉字的象形描述序列（IDS）解析为树结构，利用递归树MLP编码，并将结构嵌入融合至Transformer编码器中，不修改主干结构。在Wu等（2025）的结构探针基准测试中，CNM-BERT在长尾与未登录字上的结构准确率较最强基线（ChineseBERT）提升9.8，部件F1提升7.7。此外，该模型在CLUE、MRC和NER任务上均实现稳定增益，无论基础版还是大模型版本，表明显式注入结构信息可同时提升未登录字理解能力与下游任务表现。
原文摘要 · Abstract (English)
Token-based encoders like BERT treat Chinese characters as atomic identifiers, ignoring their recursive orthographic structure. Consequently, models rely on contextual co-occurrence, degrading performance on rare and out-of-vocabulary (OOV) characters. We propose the Compositional Network Model (CNM), a lightweight augmentation that injects discrete compositional structure into Transformer encoders. CNM parses Ideographic Description Sequences (IDS) into trees, encodes them via a recursive Tree-MLP, and fuses the structural embeddings into BERT without modifying the backbone. Evaluated on the Wu et al. (2025) structural-probing benchmark, CNM-BERT outperforms the strongest baseline (ChineseBERT) on long-tail and OOV characters by +9.8 Structure accuracy and +7.7 Radical F1. Furthermore, CNM-BERT achieves consistent gains across CLUE, MRC, and NER tasks at both base and large scales, demonstrating that explicit structural injection delivers both robust OOV understanding and tangible downstream value.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。