用低级风格变化训练通用文本风格嵌入，挑战传统假设。
Challenging Assumptions in Learning Generic Text Style Embeddings
- 通过对比学习和交叉熵损失微调通用编码器捕捉低级风格变化。
- 训练出的风格嵌入未能稳定表征高级文本风格，效果不一致。
- 适合研究风格迁移、文本生成中风格建模的学者参考。
近年来语言表示学习主要聚焦于语言建模以获取有意义的表示，常忽视风格相关考量。本文填补这一空白，构建了对风格任务至关重要的通用句级风格嵌入。方法基于一个前提：低级别文本风格变化可组合成任意高级风格。我们假设将此概念应用于表示学习，可开发出通用文本风格嵌入。通过使用对比学习和标准交叉熵损失微调通用文本编码器，旨在捕捉这些低级别风格变化，预期其能为高级文本风格提供洞察。然而结果表明，所学风格表示并不总能有效捕捉高级文本风格，促使我们重新审视底层假设。
原文摘要 · Abstract (English)
Recent advancements in language representation learning primarily emphasize language modeling for deriving meaningful representations, often neglecting style-specific considerations. This study addresses this gap by creating generic, sentence-level style embeddings crucial for style-centric tasks. Our approach is grounded on the premise that low-level text style changes can compose any high-level style. We hypothesize that applying this concept to representation learning enables the development of versatile text style embeddings. By fine-tuning a general-purpose text encoder using contrastive learning and standard cross-entropy loss, we aim to capture these low-level style shifts, anticipating that they offer insights applicable to high-level text styles. The outcomes prompt us to reconsider the underlying assumptions as the results do not always show that the learned style representations capture high-level text styles.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。