通过隐式聚类蒸馏，从日志中挖掘可解释的结构知识。
iCD: A Implicit Clustering Distillation Mathod for Structural Information Mining
- 用解耦日志的格拉姆矩阵捕捉潜在语义结构
- 在细粒度分类上提升5.08%，无需标签或特征对齐
- 适合需要模型可解释性的场景，如医疗、金融
Logit知识蒸馏因其简洁性及无需中间特征对齐而受到广泛关注；然而其决策过程可解释性有限。为此，我们提出隐式聚类蒸馏（iCD）：一种简单有效的方法，无需真实标签或特征空间对齐，即可从日志中挖掘并传递可解释的结构知识。iCD利用解耦局部日志表示上的格拉姆矩阵，使学生模型学习到潜在语义结构模式。在基准数据集上的大量实验表明，iCD在多种教师-学生架构下均有效，尤其在细粒度分类任务中表现突出，最高较基线提升5.08%。代码已公开于：https://github.com/maomaochongaa/iCD。
原文摘要 · Abstract (English)
Logit Knowledge Distillation has gained substantial research interest in recent years due to its simplicity and lack of requirement for intermediate feature alignment; however, it suffers from limited interpretability in its decision-making process. To address this, we propose implicit Clustering Distillation (iCD): a simple and effective method that mines and transfers interpretable structural knowledge from logits, without requiring ground-truth labels or feature-space alignment. iCD leverages Gram matrices over decoupled local logit representations to enable student models to learn latent semantic structural patterns. Extensive experiments on benchmark datasets demonstrate the effectiveness of iCD across diverse teacher-student architectures, with particularly strong performance in fine-grained classification tasks -- achieving a peak improvement of +5.08% over the baseline. The code is available at: https://github.com/maomaochongaa/iCD.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。