用模型自身预测构建图结构,让特征表示更符合语义
Graph Your Own Prompt
- 基于模型输出生成关系图,动态调整特征间相似性
- 提升类内凝聚度,使特征结构更清晰,泛化能力更强
- 无需改架构或训练流程,适合各类模型快速部署
我们提出图一致性正则化(GCR),一种将模型预测生成的关系图结构注入学习过程的新框架,以促进类别感知、语义有意义的特征表示。作为自提示的一种形式,GCR使模型能利用自身输出优化内部结构。深度网络虽能学习丰富表征,但常捕获与预测语义矛盾的类间噪声相似性。GCR通过在任意深度引入无参数的图一致性层(GCL),每层构建批次级特征相似性图,并将其与全局类别感知的掩码预测图对齐,该图通过调制软最大值预测相似性与类内指示器得到。这种对齐强制特征关系反映类别一致的预测行为,作为全网络的语义正则器。不同于以往工作,GCR采用多层跨空间图对齐机制与自适应加权,层重要性由图差异幅度学习得出,使模型优先选择语义可靠的层并抑制噪声层,从而在不修改架构或训练流程的情况下提升特征质量。GCR具有模型无关性、轻量性,在多种网络和数据集上均表现优异,实验证明其可促进更清洁的特征结构、更强的类内凝聚力及更好的泛化性能,为从预测结构中学习提供了新视角。
原文摘要 · Abstract (English)
We propose Graph Consistency Regularization (GCR), a novel framework that injects relational graph structures, derived from model predictions, into the learning process to promote class-aware, semantically meaningful feature representations. Functioning as a form of self-prompting, GCR enables the model to refine its internal structure using its own outputs. While deep networks learn rich representations, these often capture noisy inter-class similarities that contradict the model's predicted semantics. GCR addresses this issue by introducing parameter-free Graph Consistency Layers (GCLs) at arbitrary depths. Each GCL builds a batch-level feature similarity graph and aligns it with a global, class-aware masked prediction graph, derived by modulating softmax prediction similarities with intra-class indicators. This alignment enforces that feature-level relationships reflect class-consistent prediction behavior, acting as a semantic regularizer throughout the network. Unlike prior work, GCR introduces a multi-layer, cross-space graph alignment mechanism with adaptive weighting, where layer importance is learned from graph discrepancy magnitudes. This allows the model to prioritize semantically reliable layers and suppress noisy ones, enhancing feature quality without modifying the architecture or training procedure. GCR is model-agnostic, lightweight, and improves semantic structure across various networks and datasets. Experiments show that GCR promotes cleaner feature structure, stronger intra-class cohesion, and improved generalization, offering a new perspective on learning from prediction structure. [Project website](https://darcyddx.github.io/gcr/) [Code](https://github.com/Darcyddx/graph-prompt)
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。