将自监督学习视为教师与学生间的离散通信,用二进制信道传递语义信息。
Self-Supervised Learning as Discrete Communication
- 用二进制消息代替连续特征对齐,通过逐元素交叉熵强制离散一致
- 在图像分类、检索等任务上超越连续基线,领域迁移下仍保持性能优势
- 二进制码构成紧凑语义语言,可跨类别复用,适合研究表示结构
多数自监督学习方法通过对齐同一输入的不同视图来学习连续视觉表示,对表示维度的信息结构控制有限。本文将视觉自监督学习建模为教师与学生网络间的离散通信过程,语义信息通过容量固定的二进制信道传输。学生不直接对齐连续特征,而是预测教师生成的多标签二进制消息。通过逐元素二进制交叉熵目标实现离散一致性,同时引入编码率正则项,促使受限信道的高效利用,促进结构化表示。进一步发现,周期性重置投影头能增强此效果,使嵌入在多个离散编码中仍具预测性。大量实验表明,在图像分类、检索及密集视觉预测任务上均持续优于连续对齐基线,并在自监督适应下的领域偏移场景中表现良好。此外,分析所学二进制码发现其形成紧凑且信息丰富的离散语言,捕获可跨类别复用的语义因子。
原文摘要 · Abstract (English)
Most self-supervised learning (SSL) methods learn continuous visual representations by aligning different views of the same input, offering limited control over how information is structured across representation dimensions. In this work, we frame visual self-supervised learning as a discrete communication process between a teacher and a student network, where semantic information is transmitted through a fixed-capacity binary channel. Rather than aligning continuous features, the student predicts multi-label binary messages produced by the teacher. Discrete agreement is enforced through an element-wise binary cross-entropy objective, while a coding-rate regularization term encourages effective utilization of the constrained channel, promoting structured representations. We further show that periodically reinitializing the projection head strengthens this effect by encouraging embeddings that remain predictive across multiple discrete encodings. Extensive experiments demonstrate consistent improvements over continuous agreement baselines on image classification, retrieval, and dense visual prediction tasks, as well as under domain shift through self-supervised adaptation. Beyond backbone representations, we analyze the learned binary codes and show that they form a compact and informative discrete language, capturing semantic factors reusable across classes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。