用模糊特征融合提升语言模型可控性，生成更精准的语义内容。
Semantic Fusion with Fuzzy-Membership Features for Controllable Language Modelling
- 为每个词添加可解释的模糊语义特征，通过门控适配器融合到模型中。
- 在合成数据上降低困惑度，实现对情感极性和标点的精确控制。
- 轻量级设计，兼容现有结构，适合需要可解释生成的场景。
我们提出语义融合，一种轻量级方案，通过并行的模糊隶属度特征通道增强Transformer语言模型（LM），编码词级别语义。每个词由一组可解释特征向量表示（如词性、浅层角色、边界标志、情感极性与强度），其值来自可微分隶属函数（如幂核）。这些词级向量构成句子级语义矩阵，通过门控适配器融合进模型。训练采用标准的下一个词预测任务，一个重建语义特征的辅助损失，以及一个轻量级均匀化正则项以规范形容词类别分布。在包含两句话的合成语料上，针对未见形容词的分布外（OOD）控制，语义融合提升了困惑度，并实现了对情感极性和标点的精确用户可控生成。该方法仅增加少量开销，完全兼容共享输入输出嵌入，为条件自然语言生成提供可解释路径。
原文摘要 · Abstract (English)
We propose semantic fusion, a lightweight scheme that augments a Transformer language model (LM) with a parallel, fuzzy-membership feature channel that encodes token-level semantics. Each token is represented by a vector of interpretable features (e.g. part-of-speech cues, shallow roles, boundary flags, sentiment polarity and strength) whose values are graded degrees from differentiable membership functions (e.g. power kernels). These per-token vectors form a sentence-level semantic matrix fused via a gated adapter into the LM. Training uses standard next-token prediction, an auxiliary loss that reconstructs the semantic features from hidden states, and a lightweight uniformizer that regularizes adjective-class distributions. On a synthetic two-clause corpus with held-out adjectives for out-of-distribution (OOD) control, semantic fusion improves perplexity and enables precise, user-controllable generation of polarity and punctuation while maintaining model simplicity. This approach adds only small overhead, remains fully compatible with tied input-output embeddings, and provides an interpretable pathway for conditioned natural language generation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。