用低维表示建模感官语言与文风关系，大幅压缩模型参数
SLIM-LLMs: Modeling of Style-Sensory Language RelationshipsThrough Low-Dimensional Representations
- 用低秩岭回归提取24维文风特征，替代74维原始特征
- 在5种文体中性能接近全量模型，参数减少最高80%
- 适合追求轻量化且需解释性的自然语言分析场景
感官语言——与视觉、听觉、触觉、味觉、嗅觉及内感受相关的语言，在表达体验和感知中起着基础作用。本文通过一种新的低秩岭回归（R4）方法，探索感官语言与传统文风特征（如LIWC衡量的）之间的关系。结果表明，使用24维的低维潜在表示比74维完整特征集更有效捕捉感官语言的风格信息。为此提出徐变可解释文风模型（SLIM-LLMs），建模这些风格维度间的非线性关系。在五种文体上的评估显示，采用低秩LIWC特征的SLIM-LLMs在性能上匹配全规模语言模型，同时参数量最多减少80%。
原文摘要 · Abstract (English)
Sensorial language -- the language connected to our senses including vision, sound, touch, taste, smell, and interoception, plays a fundamental role in how we communicate experiences and perceptions. We explore the relationship between sensorial language and traditional stylistic features, like those measured by LIWC, using a novel Reduced-Rank Ridge Regression (R4) approach. We demonstrate that low-dimensional latent representations of LIWC features r = 24 effectively capture stylistic information for sensorial language prediction compared to the full feature set (r = 74). We introduce Stylometrically Lean Interpretable Models (SLIM-LLMs), which model non-linear relationships between these style dimensions. Evaluated across five genres, SLIM-LLMs with low-rank LIWC features match the performance of full-scale language models while reducing parameters by up to 80%.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。