重实现并分析测试时记忆模型 Titans，发现其并非总胜过基线。
Titans Revisited: A Lightweight Reimplementation and Critical Analysis of a Test-Time Memory Model
- 轻量级重实现 Titans 模型，验证其在多个任务上的表现。
- 记忆组件提升性能，但分块策略限制整体优势。
- 适合关注测试时学习与可复现性的研究者参考。
截至2024年底，谷歌研究人员提出 Titans：Learning at Test Time，一种在多个任务上表现优异的神经记忆模型。然而，原始描述模糊且无公开代码，导致复现困难。本文对 Titans 进行轻量级重实现，并在掩码语言建模、时间序列预测和推荐任务上进行全面评估。结果表明，由于分块机制，Titans 并非始终优于现有基线。但其神经记忆组件相较于仅使用注意力机制的模型，性能持续提升。研究证实了该模型的创新潜力，同时揭示了其实际局限性，并为未来研究提出关键问题。
原文摘要 · Abstract (English)
By the end of 2024, Google researchers introduced Titans: Learning at Test Time, a neural memory model achieving strong empirical results across multiple tasks. However, the lack of publicly available code and ambiguities in the original description hinder reproducibility. In this work, we present a lightweight reimplementation of Titans and conduct a comprehensive evaluation on Masked Language Modeling, Time Series Forecasting, and Recommendation tasks. Our results reveal that Titans does not always outperform established baselines due to chunking. However, its Neural Memory component consistently improves performance compared to attention-only models. These findings confirm the model's innovative potential while highlighting its practical limitations and raising questions for future research.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。