用分组语义原型实现高效可扩展的嵌入投影语义引导
Scalable Semantic Steering of Embedding Projections

- 将语义意图从单个样本转移到用户定义的分组,仅需一次LLM调用生成群体特征
- 在5000篇文献上达到与逐项推理相当的全局对齐效果,但调用次数减少1000倍以上
- 适用于大规模文本和多模态嵌入,适合需要高效交互式分析的科研人员
低维投影支持高维嵌入数据的交互式可视化分析,但其结构常与分析师定义的语义关系不一致。近期基于大模型的语义引导方法通过外部化用户指定种子样本组的意图来弥补这一差距,但依赖对每个项目单独进行大模型推理,导致调用次数和成本随数据量线性增长。本文提出一种可扩展的语义引导方法,将语义计算从个体项目转移至用户定义的组别。仅需一次大模型调用即可生成所有组别的结构化特征,这些特征与种子中心点结合形成混合语义原型。随后通过嵌入空间的软分配、弃权机制和缩放对齐更新,在无需重新训练的情况下传播意图并完成重投影。在包含5000篇文档的LitCovid语料库上,该方法实现了与逐项大模型引导相当的全局对齐效果,同时将大模型调用次数减少超过三个数量级。图像案例研究显示,同一原型机制可扩展至多模态嵌入。结果表明,基于分组的表示能显著提升语义引导在大规模嵌入集合中的实用性。
原文摘要 · Abstract (English)
Low-dimensional projections support interactive visual analysis of high-dimensional data embeddings, but their structure often does not align with analyst-defined semantic relationships. Recent LLM-augmented semantic steering methods address this gap by externalizing analyst intent from user-defined groups of seed examples, but they propagate intent through per-item LLM reasoning, causing LLM calls and cost to grow linearly with collection size. We propose a scalable semantic steering method that shifts semantic computation from individual items to user-defined groups. A single LLM call generates structured profiles for all groups, which are embedded and combined with seed centroids to form hybrid semantic prototypes. The method then propagates intent without retraining, using embedding-space soft assignment, abstention, and alignment-scaled updates before reprojection. On a 5K-document LitCovid corpus, our method achieves global alignment comparable to per-item LLM steering while reducing LLM calls by over three orders of magnitude. An image case study shows that the same prototype-based mechanism extends to multimodal embeddings. These results suggest that group-level representations can make semantic steering more practical for larger embedding collections.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。