arXiv:2608.15224cs.LG2026-08中稿 · publication in Tra…

用原型引导对比学习,让固定文本嵌入更精准区分语义任务。

Structuring Semantic Embeddings for Principle Evaluation: A Prototype-Guided Contrastive Learning Approach

论文配图:Structuring Semantic Embeddings for Principle Evaluation: A Prototype-Guided Contrastive Learning Approach
图 1 · 摘自论文原文
  • 基于冻结嵌入构建原型引导对比模块,通过几何正则化优化表示空间。
  • 在亚马逊评论、情感分类和毒性检测上均优于原始嵌入,尤其在评论评分任务提升显著。
  • 适合需要高精度后评估但无法微调模型的场景,如安全审查与内容过滤。

可靠的后验评估需判断生成文本是否满足目标准则。本文研究基于原则评估代理任务的冻结嵌入设定:毒性检测、细粒度情绪分类和有序评论评分。通用文本嵌入广泛用于此类任务,但语义相似性可能导致任务不同样本在表示空间中重叠。我们提出原型引导对比学习(PGCL),一种基于冻结文本嵌入的原型引导几何正则化模块。该模块融合语义流、原型锚点注意力流、监督对比学习、基于偏移的原型边界正则化及流正则化,生成紧凑的任务适配表示,无需更新基础编码器。控制实验表明,PGCL在三个数据集上均优于原始冻结嵌入,在AmazonReviews上实现最清晰的直接基线差距,同时在GoEmotions和ToxicComment上保持与强直接冻结度量学习基线相当的竞争力。我们还引入监督残差适配器、encoder-LoRA、全微调、目标消融、敏感性分析及完全记录的少样本LLM协议诊断,以界定主张边界。理论分析修订为原型映射空间下原型边界行为的充分条件说明,而非无条件训练或最终嵌入分离保证。

原文摘要 · Abstract (English)

Reliable post-hoc evaluation asks whether already generated text satisfies a target criterion after generation. In this paper we study a focused frozen-embedding setting using principle-evaluation proxy tasks: toxicity detection, fine-grained emotion categorization, and ordinal review rating. General-purpose text embeddings are widely deployed for such tasks, but broad semantic similarity can place semantically similar yet task-distinct examples in overlapping regions of the representation space. We introduce Prototype-Guided Contrastive Learning (PGCL), a prototype-guided geometric regularization module built on top of frozen text embeddings. The module combines a semantic stream, a prototype-anchor attention stream, supervised contrastive learning, offset-based prototype-margin regularization, and stream regularization to produce a compact task-adapted representation without updating the base encoder. Controlled experiments show that PGCL improves over raw frozen embeddings on all three datasets and gives the clearest direct-baseline margin on AmazonReviews, while remaining competitive with strong direct frozen metric-learning baselines on GoEmotions and ToxicComment. We also add supervised residual-adapter, encoder-LoRA, full fine-tuning, objective ablation, sensitivity, and fully logged few-shot LLM protocol diagnostics to define the boundary of the claim. The theoretical analysis is revised as a sufficient-condition account for prototype-margin behavior under explicit assumptions in the prototype-mapping space, rather than as an unconditional training or final-embedding separation guarantee.

嵌入优化对比学习后验评估冻结模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。