用冻结嵌入+递归结构，实现低资源语言质量评估的高参数效率
Parameter-Efficient Quality Estimation via Frozen Recursive Models
- 采用冻结的XLM-R嵌入与递归模型结合，减少参数量
- 在8个语言对上达到0.370的斯皮尔曼相关系数，媲美全量微调
- 参数量减少80倍，适合资源受限场景下的质量评估应用
Tiny Recursive Models (TRM) 通过共享网络的迭代优化，在推理任务中表现优异。本文探究其递归机制是否适用于低资源语言的质量评估（QE），采用三阶段方法。在包含8个语言对的低资源QE数据集上，实验发现：递归机制无法有效迁移至QE，外部迭代反而降低性能，内部递归仅带来微弱收益；表示质量远超架构设计；冻结预训练嵌入可达到与微调相当的效果，同时将可训练参数减少37倍（7M vs 262M）。使用冻结XLM-R嵌入的TRM-QE在测试中获得0.370的斯皮尔曼相关系数，与全微调版本（0.369）相当，并优于同深度的标准Transformer（0.336）。在印地语和泰米尔语上，冻结版TRM-QE超越560M参数的MonoTransQuest，且可训练参数仅为其1/80，表明权重共享结合冻结嵌入能实现高质量评估的极高参数效率。代码已开源。
原文摘要 · Abstract (English)
Tiny Recursive Models (TRM) achieve strong results on reasoning tasks through iterative refinement of a shared network. We investigate whether these recursive mechanisms transfer to Quality Estimation (QE) for low-resource languages using a three-phase methodology. Experiments on $8$ language pairs on a low-resource QE dataset reveal three findings. First, TRM's recursive mechanisms do not transfer to QE. External iteration hurts performance, and internal recursion offers only narrow benefits. Next, representation quality dominates architectural choices, and lastly, frozen pretrained embeddings match fine-tuned performance while reducing trainable parameters by 37$\times$ (7M vs 262M). TRM-QE with frozen XLM-R embeddings achieves a Spearman's correlation of 0.370, matching fine-tuned variants (0.369) and outperforming an equivalent-depth standard transformer (0.336). On Hindi and Tamil, frozen TRM-QE outperforms MonoTransQuest (560M parameters) with 80$\times$ fewer trainable parameters, suggesting that weight sharing combined with frozen embeddings enables parameter efficiency for QE. We release the code publicly for further research. Code is available at https://github.com/surrey-nlp/TRMQE.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。