用大模型内部状态做语义聚类,提速推理且不丢效果
Efficient Latent Semantic Clustering for Scaling Test-Time Computation of LLMs
- 直接利用大模型隐状态聚类,无需外部模型
- 效率提升显著,性能持平或更好
- 适合需要高效推理的部署场景
扩展测试时计算——对单个输入生成并分析多个或连续输出——已成为提升大语言模型(LLMs)可靠性与质量的有力策略,体现在不确定性量化和多步推理的进步中。其核心环节是语义聚类,即对形式不同但含义相同的输出进行分组。语义聚类可估计输出语义分布,避免重复探索推理路径。然而现有方法通常依赖外部模型,带来巨大计算开销,且难以捕捉上下文相关的语义。我们提出潜空间语义聚类(LSC),一种轻量级、上下文感知的方法,利用生成式大模型内部隐藏状态进行聚类,无需外部模型。在多种大模型和数据集上的广泛实验表明,LSC 显著提升了测试时扩展计算的效率,同时保持或超越了现有方法的性能。
原文摘要 · Abstract (English)
Scaling test-time computation--generating and analyzing multiple or sequential outputs for a single input--has become a promising strategy for improving the reliability and quality of large language models (LLMs), as evidenced by advances in uncertainty quantification and multi-step reasoning. A key shared component is semantic clustering, which groups outputs that differ in form but convey the same meaning. Semantic clustering enables estimation of the distribution over the semantics of outputs and helps avoid redundant exploration of reasoning paths. However, existing approaches typically rely on external models, which introduce substantial computational overhead and often fail to capture context-aware semantics. We propose Latent Semantic Clustering (LSC), a lightweight and context-sensitive method that leverages the generator LLM's internal hidden states for clustering, eliminating the need for external models. Our extensive experiment across various LLMs and datasets shows that LSC significantly improves the computational efficiency of test-time scaling while maintaining or exceeding the performance of existing methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。