对比了文档微调模型与基础模型在心理评估中的表现，发现前者更准确且鲁棒。
Evaluating Document-Tuned Transformer Representations for Person-level Mental Health Assessment
- 用相同架构对比基础模型和文档微调模型的性能差异。
- 文档微调模型相关性提升13.4%（p=0.015），预测更准。
- 更适合处理含不确定表达的文本，适合心理状态分析场景。
个体心理评估需要整合同一人多条消息的语义信息，而文档级训练目标并非为此设计。本文在相同条件下，系统性地对比了架构一致的基线变换器与文档微调变换器（进一步在文档层级进行对比学习微调，常称“句子变换器”）。在两个纵向心理健康数据集上比较层间与整体性能，发现文档微调模型相比基础表示显著提升（皮尔逊相关系数提高13.4%，p=0.015）。鲁棒性分析显示，文档微调模型在词删除、同义词替换、打字错误和反向翻译等扰动下仍保持更高精度。此外，文档微调嵌入更能捕捉含糊语言（如‘通常’）特征，而基础模型更倾向编码数量表达（如‘很多’），表明文档微调模型对不确定性建模更优。结果说明表示选择影响心理状态预测，文档微调模型通常更具优势。
原文摘要 · Abstract (English)
Person-level psychological assessment requires aggregating meaning across many messages from the same individual, a task that document-level training objectives were not explicitly designed for. We present a systematic, empirical comparison between architecturally matched traditional (a) base-transformers and (b) document-tuned-transformers (further contrastively fine-tuned at the document-level, sometimes referred to as "sentence transformers") under otherwise identical conditions. Comparing layer-wise and overall performance across two longitudinal mental health and psychological datasets, we find document-tuned models demonstrated a consistent improvement over base representations (increase in Pearson r of 13.4%, p=.015). Robustness analyses revealed document-tuned models remained more accurate under perturbations to word deletion, synonym replacement, typo injection, and back translation. Further, hedged language (e.g., `usually') was more characteristic of outcomes in document-tuned embeddings while abundance (e.g., `lot') was more characteristic of base-transformers, suggesting document-tuned models may better capture uncertainty. These results suggest representation choice impacts mental health prediction, document-tuned models often being more adept.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。