用三轴张力模型定位并控制文生图中的幻觉问题
Taming the Tri-Space Tension: ARC-Guided Hallucination Modeling and Control for Text-to-Image Generation
- 将幻觉视为潜空间中的三轴对齐张力,提出动态量化指标ARC
- ARC信号可实时识别幻觉主因,精度提升32%以上
- 轻量控制器在生成中干预特定轴,不损失图像质量
尽管文生图扩散模型在图像质量和提示保真度上取得显著进展,仍普遍存在'幻觉'现象,即生成内容与意图提示语义存在细微或显著偏差。我们提出一种认知启发视角,将幻觉重新诠释为潜空间对齐轨迹的偏移。实证发现生成过程处于多轴认知张力场中,需持续权衡语义连贯性、结构对齐和知识锚定三个关键轴。我们将其形式化为幻觉三轴空间(Hallucination Tri-Space),并引入动态向量表示——对齐风险码(ARC):其幅值表征整体错位程度,方向指示主导失配轴,不平衡度反映张力不对称性。基于此,我们设计了仅在潜空间操作的轻量级控制器TensionModulator (TM-ARC),通过监测ARC信号,在采样过程中进行轴向针对性干预。在标准文生图基准测试中,该方法显著降低幻觉率,同时保持图像质量与多样性。该框架为理解与缓解扩散模型生成失败提供了统一且可解释的方案。
原文摘要 · Abstract (English)
Despite remarkable progress in image quality and prompt fidelity, text-to-image (T2I) diffusion models continue to exhibit persistent "hallucinations", where generated content subtly or significantly diverges from the intended prompt semantics. While often regarded as unpredictable artifacts, we argue that these failures reflect deeper, structured misalignments within the generative process. In this work, we propose a cognitively inspired perspective that reinterprets hallucinations as trajectory drift within a latent alignment space. Empirical observations reveal that generation unfolds within a multiaxial cognitive tension field, where the model must continuously negotiate competing demands across three key critical axes: semantic coherence, structural alignment, and knowledge grounding. We then formalize this three-axis space as the Hallucination Tri-Space and introduce the Alignment Risk Code (ARC): a dynamic vector representation that quantifies real-time alignment tension during generation. The magnitude of ARC captures overall misalignment, its direction identifies the dominant failure axis, and its imbalance reflects tension asymmetry. Based on this formulation, we develop the TensionModulator (TM-ARC): a lightweight controller that operates entirely in latent space. TM-ARC monitors ARC signals and applies targeted, axis-specific interventions during the sampling process. Extensive experiments on standard T2I benchmarks demonstrate that our approach significantly reduces hallucination without compromising image quality or diversity. This framework offers a unified and interpretable approach for understanding and mitigating generative failures in diffusion-based T2I systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。