用单句检测19种人类价值观，提升AI对道德判断的理解能力。
Human Values in a Single Sentence: Moral Presence, Hierarchies, and Transformer Ensembles on the Schwartz Continuum
- 基于DeBERTa的集成模型融合上下文与道德词典，提升识别精度
- 在仅8GB显存下实现0.332的宏平均F1，优于现有最佳基线
- 首次系统对比直接预测与分层筛选，为高效伦理建模提供实证依据
我们研究了在约7.4万条新闻和政治纲领文本的句子中，对精细版施瓦茨价值观量表中的19种人类价值观进行句级检测（ValueEval'24语料库）。每条句子标注了价值是否存在，形成二分类道德存在标签和19类多标签任务，且存在严重类别不平衡。首先，我们证明单句即可学习道德存在：DeBERTa-base分类器在校准阈值下达到正类F1=0.74。其次，在仅配备单张消费级8GB显存的条件下，比较直接多标签检测与存在门控层级结构，发现门控机制未带来提升，表明门控召回成为瓶颈。第三，我们探索轻量辅助信号——短程上下文、LIWC-22及道德词典，以及小型集成模型。最优监督配置为融合这些信号的DeBERTa模型软投票集成，宏观F1达0.332，超越该语料库上此前最佳英文基线（官方ValueEval'24最佳结果为0.28）。方法论上，本研究首次系统比较直接与存在门控架构、轻量特征增强编码器及中等规模指令微调大模型（7–9B）在句级精细施瓦茨价值观检测中的表现。我们还评估了Gemma 2 9B、Llama 3.1 8B、Mistral 8B、Qwen 2.5 7B等7–9B指令微调模型在零样本/少样本及QLoRA设置下的表现，发现其在相同算力预算下仍落后于监督集成模型。总体而言，研究为构建高效、具备价值观感知能力的NLP模型提供了实证指导。
原文摘要 · Abstract (English)
We study sentence-level detection of the 19 human values in the refined Schwartz continuum in about 74k English sentences from news and political manifestos (ValueEval'24 corpus). Each sentence is annotated with value presence, yielding a binary moral-presence label and a 19-way multi-label task under severe class imbalance. First, we show that moral presence is learnable from single sentences: a DeBERTa-base classifier attains positive-class F1 = 0.74 with calibrated thresholds. Second, we compare direct multi-label value detectors with presence-gated hierarchies in a setting where only a single consumer-grade GPU with 8 GB of VRAM is available, and we explicitly choose all training and inference configurations to fit within this budget. Presence gating does not improve over direct prediction, indicating that gate recall becomes a bottleneck. Third, we investigate lightweight auxiliary signals - short-range context, LIWC-22, and moral lexica - and small ensembles. Our best supervised configuration, a soft-voting ensemble of DeBERTa-based models enriched with such signals, reaches macro-F1 = 0.332 on the 19 values, improving over the best previous English-only baseline on this corpus, namely the best official ValueEval'24 English run (macro-F1 = 0.28 on the same 19-value test set). Methodologically, our study provides, to our knowledge, the first systematic comparison of direct versus presence-gated architectures, lightweight feature-augmented encoders, and medium-sized instruction-tuned Large Language Models (LLMs) for refined Schwartz values at sentence level. We additionally benchmark 7-9B instruction-tuned LLMs (Gemma 2 9B, Llama 3.1 8B, Mistral 8B, Qwen 2.5 7B) in zero-/few-shot and QLoRA setups, and find that they lag behind the supervised ensemble under the same compute budget. Overall, our results provide empirical guidance for building compute-efficient, value-aware NLP models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。