不同学习路径导致记忆遗忘程度不同,文本输入更易被遗忘。
When the Same Musical Knowledge Forgets Differently: A Clean Probe of Pathway-Dependent Forgetting

- 设计双路径对照实验,分离学习方式对遗忘的影响。
- 文本路径知识比音频路径更易遗忘,差异稳定且显著。
- 适合研究多模态模型记忆机制或系统设计的学者。
模型可通过听音乐或读文字理解《致爱丽丝》的平静抒情,但这种知识的获取路径是否影响其遗忘速度?当前多模态模型的遗忘研究未关注学习路径的影响,我们称之为路径不变假设。音乐理解提供了清晰测试场景:同一音乐片段与标准文字描述可对齐至相同感知内容,使相同知识单元通过听觉或视觉路径进入模型。在多个架构不同的音文模型中,我们观察到一致的不对称性:在相同适应压力下,文本路径知识比音频路径知识更容易遗忘。为排除混淆因素,提出配对路径受控协议(PPCP),包含三阶段设计:建立匹配路径基线,对同一批知识池进行对称监督激活双路径,施加相同的遗忘压力。该差异在不同模型间保持稳定,且在增益控制分析、反向覆盖替换为跨域正确标签学习、单模态压力下依然存在,轻量回放也无法消除。两个独立的路由深度控制验证该效应并非由架构深度导致,凸显输入表示的关键作用。在PPCP下,结果表明遗忘高度依赖路径,确立了学习路径作为遗忘研究和多模态系统设计的新分析维度。
原文摘要 · Abstract (English)
A model can learn that the piano piece Für Elise is calm and reflective by listening to the audio or by reading a text description, but does it matter which route that knowledge took when it is later at risk of being forgotten? Forgetting research in multimodal models measures what knowledge is lost under adaptation, yet has not asked whether acquisition route affects how easily that knowledge is forgotten. We call this untested premise the Pathway-Invariant Assumption. Music understanding enables a clean test because a music clip and a canonical text description can be aligned to the same perceptual content, allowing the same knowledge unit to enter a model through listening or reading while the target remains fixed. Across multiple architecturally distinct audio-language models, we observe a consistent asymmetry: text-pathway knowledge is forgotten more than matched audio-pathway knowledge under identical adaptation pressure. To attribute this effect to route rather than confounds, we introduce the Paired Pathway Controlled Protocol (PPCP), a three-phase design that establishes matched pathway baselines, activates both pathways under symmetric supervision on the same knowledge pool, and applies identical forgetting pressure to both pathways. The gap is stable across models and gain-controlled analyses, persists when contradictory overwrite is replaced by correct-label cross-domain learning, remains under single-modality pressure, and is not removed by lightweight replay. Two independent routing-depth controls confirm that the effect is not explained by architectural depth, pointing to input representation as the dominant factor. Under PPCP, our results demonstrate that forgetting is highly route-dependent, establishing acquisition route as a new analytical dimension for forgetting research and multimodal system design.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。