让普通大模型具备精准语义分解能力，无需重新训练。
Sense Representations Are Inducible Interfaces
- 通过门控残差添加，在冻结模型中注入显式语义路径。
- 零样本词义消歧达64.95 F1，跨语言迁移准确率0.988。
- 适合需要可控生成、多语言适配的工程师与研究者。
语义表征（即每个词元的显式语义分解）对消歧、可控生成和跨语言对齐很有用，但现有方法需在预训练时嵌入语义结构。我们提出ACROS，通过门控残差加法将显式语义路径引入冻结的预训练解码器语言模型。在SmolLM2-360M上，ACROS保持基础模型性能的同时，支持三类应用：零样本词义消歧（Raganato ALL数据集上F1达64.95，媲美词典首义启发式方法）、低KL值词汇操控（覆盖5,161个CoInCo案例，简单代理可恢复约90%正向变化）、SENSIA跨语言适应（四语言平均R@1为0.988，目标FLORES PPL为7.94）。ACROS使语义表征成为普通预训练语言模型可诱导的接口。
原文摘要 · Abstract (English)
Sense representations (explicit, per-token meaning decompositions) are useful for disambiguation, steering, and cross-lingual alignment, but existing approaches require models to be pretrained with sense structure baked in. We introduce ACROS, which induces an explicit sense pathway into a frozen pretrained decoder LM through a gated residual addition. On SmolLM2-360M, ACROS preserves base LM quality while supporting three uses of the same induced variables: zero-shot word-sense disambiguation (64.95 F1 on Raganato ALL, competitive with the WordNet first-sense heuristic), low-KL lexical steering across 5,161 CoInCo cases where a simple non-oracle proxy recovers about 90% of positive shifts, and SENSIA cross-lingual adaptation to four languages (mean R@1 0.988, target FLORES PPL 7.94). ACROS makes sense representations an inducible interface for ordinary pretrained LMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。