将混合嵌入逆向生成可读文本,实现可控且可解释的数据增强。
inversedMixup: Data Augmentation via Inverting Mixed Embeddings
- 通过逆映射将混合嵌入还原为自然语言,结合混合法的可控性与LLM生成的可读性。
- 首次在文本Mixup中实证发现流形入侵现象,且可通过三阶段流程缓解。
- 适用于少样本和全监督场景,提升数据增强效果,适合需要可解释性的研究者。
Mixup通过线性插值输入和标签生成增强样本,但因其在嵌入空间操作,结果不可读。相比之下,基于大模型的增强方法在标记层面生成句子,可读性强但控制能力有限。受最近大模型逆映射技术启发(从嵌入重建文本),我们提出inversedMixup:将任务模型输出嵌入空间与大模型输入嵌入空间对齐,使混合嵌入可在可控混合比例下被重构为人类可读的句子。该可解释性首次为文本Mixup中的流形入侵现象提供了实证支持。在此基础上,我们设计了三阶段数据增强方法,并引入简单有效策略缓解流形入侵。大量实验表明,该方法在少样本与全监督场景中均具有效性与通用性。
原文摘要 · Abstract (English)
Mixup generates augmented samples by linearly interpolating inputs and labels with a controllable ratio. However, since it operates at the latent embedding level, the resulting samples are not human-interpretable. In contrast, LLM-based augmentation methods produce sentences via prompts at the token level, yielding readable outputs but offering limited control over the generation process. Inspired by recent advances in LLM inversion, which reconstructs natural language from embeddings and helps bridge the gap between latent embedding space and discrete token space, we propose inversedMixup, a unified framework that combines the controllability of Mixup with the interpretability of LLM-based generation. Specifically, inversedMixup aligns the output embedding space of a task-specific model with the input embedding space of an LLM, so that mixed embeddings can be reconstructed, under a controllable mixing ratio, into human-interpretable sentences. This interpretability provides the first empirical evidence of the manifold intrusion phenomenon in text Mixup. Building on this, we extend inversedMixup into a three-stage data augmentation method, and introduce a simple yet effective strategy to mitigate manifold intrusion during augmentation. Extensive experiments demonstrate the effectiveness and generalizability of our approach in both few-shot and fully supervised scenarios.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。