用大模型隐状态分析作者心理特质，发现平均词隐状态效果最佳
Evaluation of LLMs-based Hidden States as Author Representations for Psychological Human-Centered NLP Tasks
- 用词元隐状态均值表示文档，性能最优
- 用户级隐状态虽单独使用不佳，但能提升整体表现
- 适合关注作者心理建模的人类中心NLP研究者
大多数面向人类的自然语言处理任务依赖基于Transformer的大语言模型（LLM）的隐藏状态作为表示。然而，用于表示的模型组件差异较大。同时，亟需能隐式建模作者、提供用户级隐藏状态的人类语言模型（HuLM）。本文系统评估了不同LLM与HuLM架构下，通过不同方式表示文档和用户，以预测情绪效价、唤醒度、共情力和痛苦程度等动态变化状态及稳定特质。结果表明，将文档表示为词元隐藏状态的平均值，在多数情况下表现最佳；尽管用户级隐藏状态本身并非最优，但将其融入模型可显著增强基于词元或文档的嵌入表示，从而实现最佳性能。
原文摘要 · Abstract (English)
Like most of NLP, models for human-centered NLP tasks -- tasks attempting to assess author-level information -- predominantly use representations derived from hidden states of Transformer-based LLMs. However, what component of the LM is used for the representation varies widely. Moreover, there is a need for Human Language Models (HuLMs) that implicitly model the author and provide a user-level hidden state. Here, we systematically evaluate different ways of representing documents and users using different LM and HuLM architectures to predict task outcomes as both dynamically changing states and averaged trait-like user-level attributes of valence, arousal, empathy, and distress. We find that representing documents as an average of the token hidden states performs the best generally. Further, while a user-level hidden state itself is rarely the best representation, we find its inclusion in the model strengthens token or document embeddings used to derive document- and user-level representations resulting in best performances.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。