arXiv:2602.14419cs.CLcs.LG2026-02被引 1

用傅里叶变换构建语义层级结构,解决大模型幻觉问题

WavePhaseNet: A DFT-Based Method for Constructing Semantic Conceptual Hierarchy Structures (SCHS)

  • 通过DFT分解序列,分离全局语义与局部语法
  • 将24576维降维至3000维仍保留语义完整性
  • 用上同调理论量化并抑制推理不一致性

本文基于测度论与频域分析重新诠释大语言模型中的Transformer/注意力机制,理论证明幻觉是结构性必然产物。嵌入空间作为σ代数上的条件期望,其无法与语义真值集同构,导致逻辑一致性崩溃。提出WavePhaseNet方法,利用离散傅里叶变换(DFT)沿序列维度构建语义概念层次结构(SCHS),低频成分捕捉全局语义与意图,高频成分表示局部语法与表达。通过累积能量分析发现,GPT-4的24,576维嵌入空间具有1/f谱特性,约3,000维为完整表征下限。基于重叠局部窗口的上同调正则化构建图结构与上链复形,以边界算子损失量化局部推理不一致性。结合霍奇理论的调和投影,将上同调转化为可计算的正则化原则,实现语义一致性的精确控制。

原文摘要 · Abstract (English)

This paper reformulates Transformer/Attention mechanisms in Large Language Models (LLMs) through measure theory and frequency analysis, theoretically demonstrating that hallucination is an inevitable structural limitation. The embedding space functions as a conditional expectation over a σ-algebra, and its failure to be isomorphic to the semantic truth set fundamentally causes logical consistency breakdown. WavePhaseNet Method The authors propose WavePhaseNet, which explicitly constructs a Semantic Conceptual Hierarchy Structure (SCHS) using Discrete Fourier Transform (DFT). By applying DFT along the sequence dimension, semantic information is decomposed into frequency bands: low-frequency components capture global meaning and intent, while high-frequency components represent local syntax and expression. This staged separation enables precise semantic manipulation in diagonalized space. Dimensionality Reduction GPT-4's 24,576-dimensional embedding space exhibits a 1/f spectral structure based on language self-similarity and Zipf's law. Through cumulative energy analysis, the authors derive that approximately 3,000 dimensions constitute the lower bound for "complete representation." This demonstrates that reduction from 24,576 to 3,000 dimensions preserves meaning and intent while enabling rigorous reasoning and suppressing hallucination. Cohomological Consistency Control The reduced embedding space, constructed via cohomological regularization over overlapping local windows, allows defining a graph structure and cochain complex. This quantifies inconsistencies among local inferences as coboundary-based losses. Applying harmonic projection based on Hodge theory positions cohomology as a computable regularization principle for controlling semantic consistency, extracting maximally consistent global representations.

大模型幻觉语义结构傅里叶变换上同调

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。