arXiv:2601.17495cs.LGcs.AI2026-01

用少量标签优化文本嵌入,让相似搜索更准。

PEARL: Prototype-Enhanced Alignment for Label-Efficient Representation Learning with Deployment-Driven Insights from Digital Governance Communication Systems

  • 用原型引导嵌入对齐,不改变维度也不压缩
  • 标签极少时邻居质量提升25.7%以上
  • 适合标注少、需稳定检索的政务系统

在许多已部署系统中,新文本通过检索历史案例来处理,例如数字治理平台中公民消息的路由与回复。系统失效常因嵌入空间中的最近邻对应错误案例。现代系统依赖大模型生成的固定高维嵌入,但实际部署中标签稀缺、领域漂移,重训练基础编码器成本高或不可行。下游性能高度依赖嵌入几何结构,而原始嵌入往往与最近邻检索所需的局部结构不一致。我们提出PEARL(原型增强对齐表示学习),一种标签高效的对齐方法,利用有限监督将嵌入软性对齐至类别原型。该方法重塑局部邻域结构,同时保持维度不变,避免剧烈投影或坍缩。目标是弥合纯无监督后处理(效果有限且不稳定)与全监督投影(需大量标注)之间的差距。我们在从极端标签稀缺到高标签设置的控制条件下评估PEARL。在标签稀缺场景下,PEARL显著提升局部邻域质量,相比原始嵌入提升25.7%,优于强无监督后处理超过21.1%,恰在相似性系统最脆弱的区域实现突破。

原文摘要 · Abstract (English)

In many deployed systems, new text inputs are handled by retrieving similar past cases, for example when routing and responding to citizen messages in digital governance platforms. When these systems fail, the problem is often not the language model itself, but that the nearest neighbors in the embedding space correspond to the wrong cases. Modern machine learning systems increasingly rely on fixed, high-dimensional embeddings produced by large pretrained models and sentence encoders. In real-world deployments, labels are scarce, domains shift over time, and retraining the base encoder is expensive or infeasible. As a result, downstream performance depends heavily on embedding geometry. Yet raw embeddings are often poorly aligned with the local neighborhood structure required by nearest-neighbor retrieval, similarity search, and lightweight classifiers that operate directly on embeddings. We propose PEARL (Prototype-Enhanced Aligned Representation Learning), a label-efficient approach that uses limited supervision to softly align embeddings toward class prototypes. The method reshapes local neighborhood geometry while preserving dimensionality and avoiding aggressive projection or collapse. Its aim is to bridge the gap between purely unsupervised post-processing, which offers limited and inconsistent gains, and fully supervised projections that require substantial labeled data. We evaluate PEARL under controlled label regimes ranging from extreme label scarcity to higher-label settings. In the label-scarce condition, PEARL substantially improves local neighborhood quality, yielding 25.7% gains over raw embeddings and more than 21.1% gains relative to strong unsupervised post-processing, precisely in the regime where similarity-based systems are most brittle.

嵌入对齐小样本学习相似搜索政务系统

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。