arXiv:2502.05553cs.CL2025-02

用随机嵌入变化让大模型生成更灵活多样的文本。

Latent Structure Modulation in Large Language Models Through Stochastic Concept Embedding Transitions

  • 引入随机嵌入更新机制,动态调整词向量
  • 提升低频词保留率与句子结构多样性
  • 适合需要高质量多样生成的场景

随机嵌入转换通过推理过程中的概率化更新机制,动态调节词元表示,缓解静态或确定性嵌入带来的约束。提出一种过渡框架,使每个词嵌入在生成过程中经历概率性演化,在保持语义一致性的同时增强适应性。实证评估显示,采用随机过渡的模型展现出更高的词汇多样性、更强的生成连贯性以及对低频词汇更好的保留能力,推动更丰富的句式结构,并减少对高概率词的选择依赖。跨变压器层的嵌入漂移统计分析表明,表示在保持连贯性的前提下具备更高灵活性,支持可控随机性促进上下文敏感表征学习的假设。实验结果还表明,概率嵌入带来微小计算开销却维持生成效率,具备大规模应用可行性。与传统嵌入方法的对比研究揭示了在文本补全准确率、对话连贯性和结构复杂度上的显著提升,验证了随机过渡在增强表征表达力方面的有效性。嵌入空间中的聚类模式显示,概率更新既保持了有意义的语义分组,又实现上下文驱动的迁移,进一步证实该机制的稳定性。性能指标表明,随机过渡在适应性与控制之间取得平衡,确保生成内容语言连贯且不过度随机。

原文摘要 · Abstract (English)

Stochastic embedding transitions introduce a probabilistic mechanism for adjusting token representations dynamically during inference, mitigating the constraints imposed through static or deterministic embeddings. A transition framework was proposed in which each token embedding evolved through probabilistic updates, ensuring adaptability while preserving semantic integrity across linguistic contexts. Empirical evaluations demonstrated that models incorporating stochastic transitions exhibited greater lexical diversity, improved generative coherence, and enhanced retention of low-frequency vocabulary, contributing to more varied sentence structures and reduced reliance on high-probability token selections. Statistical analyses of embedding drift across transformer layers indicated that representations evolved more flexibly without losing coherence, supporting the hypothesis that controlled stochasticity facilitated context-sensitive representation learning. Experimental results revealed that probabilistic embeddings introduced minor computational overhead while maintaining generative efficiency, reinforcing their feasibility in large-scale applications. A comparative study with traditional embedding approaches highlighted measurable gains in text completion accuracy, dialogue coherence, and structural complexity, confirming the effectiveness of stochastic transitions in enhancing representation expressiveness. Clustering patterns in the embedding space suggested that probabilistic updates preserved meaningful semantic groupings while enabling context-driven shifts, further validating the stability of the transition mechanism. Performance metrics indicated that stochastic transitions balanced adaptability and control, ensuring that generative outputs remained linguistically coherent without excessive randomness.

大模型嵌入优化生成多样性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。