用反事实生成多维度内容质量评估框架，提升与人工判断的相关性。
Multi-Facet Counterfactual Learning for Content Quality Evaluation
- 通过大模型生成对比原文的反事实内容，模拟不同质量维度变化
- 结合对比学习与监督学习，使评估器能区分多个质量维度
- 在两个数据集上验证，显著提升评估结果与人工判断的一致性
文档质量评估对于从海量信息中筛选有价值内容至关重要。传统方法通常依赖单一评分作为训练监督信号，难以区分文档在多个质量维度上的差异。本文提出多维度反事实学习（MOLE）框架，高效构建能感知多维度内容质量的评估器。给定特定场景，我们利用大语言模型生成与原文在关键质量维度上存在差异的反事实内容。同时，采用基于对比学习与监督学习的联合训练策略，使评估器能够有效区分不同质量维度，从而更准确预测内容质量得分。在两个不同场景下的数据集上的实验结果表明，所提MOLE框架显著提升了文档质量评估与人工判断的一致性，为高效信息获取提供了有力工具。
原文摘要 · Abstract (English)
Evaluating the quality of documents is essential for filtering valuable content from the current massive amount of information. Conventional approaches typically rely on a single score as a supervision signal for training content quality evaluators, which is inadequate to differentiate documents with quality variations across multiple facets. In this paper, we propose Multi-facet cOunterfactual LEarning (MOLE), a framework for efficiently constructing evaluators that perceive multiple facets of content quality evaluation. Given a specific scenario, we prompt large language models to generate counterfactual content that exhibits variations in critical quality facets compared to the original document. Furthermore, we leverage a joint training strategy based on contrastive learning and supervised learning to enable the evaluator to distinguish between different quality facets, resulting in more accurate predictions of content quality scores. Experimental results on 2 datasets across different scenarios demonstrate that our proposed MOLE framework effectively improves the correlation of document content quality evaluations with human judgments, which serve as a valuable toolkit for effective information acquisition.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。