用重建目标自适应划分文本，让压缩更高效且保留主题信息
ReconSpan: Reconstruction-Guided Adaptive Latent Tokenization
- 根据重建效果动态分块，每块由前缀编码还原
- 平均块长6.5~12.2，比随机分块多保留30%以上文本
- 适合需要主题理解但不需精确细节的场景
自适应潜在标记化将细粒度输入映射为较短的、与输入相关的连续表示序列。本文提出ReconSpan，通过后向解码器从单一上下文前缀编码中重建文本块，并为每个块保留一个这样的编码作为潜在标记。在分块过程中应用重建准则，使单个训练好的自编码器可实现6.5至12.2的平均块长。在相同平均长度下，基于重建引导的边界比随机边界保留更多文本内容。生成的潜在序列能可靠恢复主题信息，但在提取精确细节时表现较差。
原文摘要 · Abstract (English)
Adaptive latent tokenization maps a fine-grained input to a shorter sequence of continuous representations associated with input-dependent spans. We introduce ReconSpan, which divides text into chunks that a backward decoder can reconstruct from a single contextual prefix code and retains one such code as the latent token for each chunk. The reconstruction criterion is applied when chunks are formed, allowing one trained autoencoder to produce average chunk lengths from 6.5 to 12.2. At matched average length, reconstruction-guided boundaries preserve more text than random boundaries. Readers of the resulting latent sequence recover topic information reliably but struggle to extract exact details.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。