arXiv:2603.26801cs.LGcs.AI2026-03

一种统一的稀疏机制,让多模态模型更高效且更可靠。

Sparse-by-Design Cross-Modality Prediction: L0-Gated Representations for Reliable and Efficient Learning

  • 用硬-混凝土门控直接在表示层施加稀疏,跨模态通用。
  • 在三个数据集上激活更少特征,同时降低校准误差。
  • 适合需要可比效率与可靠性的多模态模型部署者。

预测系统越来越多地处理异构模态,如图、文本和表格数据,但稀疏化与效率优化仍为模态特异性(如图边/邻域稀疏、Transformer头/层剪枝、独立的表格特征选择流程)。这种碎片化导致结果难以比较,部署复杂,且削弱了端到端KDD流程中的可靠性分析。一个统一的稀疏化原语将使不同模态间的准确率-效率权衡可比,并支持在表示压缩下的可控可靠性分析。我们提出L0-Gated Cross-Modality Learning(L0GM),一种模态无关、特征级的硬-混凝土门控框架,直接在学习到的表示上施加L0式稀疏。L0GM在各模态的分类器接口处附加硬-混凝土随机门控:节点嵌入(GNNs)、池化序列嵌入(如CLS,Transformer)和学习到的表格嵌入向量(表格模型)。实现端到端可训练稀疏化,并提供对活跃特征比例的显式控制。为稳定优化并使权衡可解释,引入L0退火调度,生成清晰的准确率-稀疏帕累托前沿。在ogbn-products、Adult、IMDB三个公开基准上,L0GM实现有竞争力的预测性能,同时激活更少的表示维度,并降低期望校准误差(ECE)。总体而言,L0GM建立了一个模态无关、可复现的稀疏化原语,支持跨异构模态的准确率、效率与校准权衡分析。

原文摘要 · Abstract (English)

Predictive systems increasingly span heterogeneous modalities such as graphs, language, and tabular records, but sparsity and efficiency remain modality-specific (graph edge or neighborhood sparsification, Transformer head or layer pruning, and separate tabular feature-selection pipelines). This fragmentation makes results hard to compare, complicates deployment, and weakens reliability analysis across end-to-end KDD pipelines. A unified sparsification primitive would make accuracy-efficiency trade-offs comparable across modalities and enable controlled reliability analysis under representation compression. We ask whether a single representation-level mechanism can yield comparable accuracy-efficiency trade-offs across modalities while preserving or improving probability calibration. We propose L0-Gated Cross-Modality Learning (L0GM), a modality-agnostic, feature-wise hard-concrete gating framework that enforces L0-style sparsity directly on learned representations. L0GM attaches hard-concrete stochastic gates to each modality's classifier-facing interface: node embeddings (GNNs), pooled sequence embeddings such as CLS (Transformers), and learned tabular embedding vectors (tabular models). This yields end-to-end trainable sparsification with an explicit control knob for the active feature fraction. To stabilize optimization and make trade-offs interpretable, we introduce an L0-annealing schedule that induces clear accuracy-sparsity Pareto frontiers. Across three public benchmarks (ogbn-products, Adult, IMDB), L0GM achieves competitive predictive performance while activating fewer representation dimensions, and it reduces Expected Calibration Error (ECE) in our evaluation. Overall, L0GM establishes a modality-agnostic, reproducible sparsification primitive that supports comparable accuracy, efficiency, and calibration trade-off analysis across heterogeneous modalities.

稀疏学习多模态模型压缩概率校准

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。