首个德语文本作者验证基准,测试风格随体裁、时间与AI写作的变化
When Writing Style Drifts: Benchmarking Authorship Verification under Distribution Shifts in Genre, Time and the AI-Era

- 构建跨体裁、跨时序的德语文本对数据集,系统评估作者验证模型鲁棒性
- 时序差异导致性能显著下降,但当前AI写作未引发明显风格漂移
- 细调大模型在跨体裁验证中表现最佳,特定风格特征在不同体裁中重要性不同
作者身份验证(AV)假设作者写作风格足够稳定,可区分于他人。然而现实中,体裁、时间变化及AI辅助写作带来的分布偏移挑战了这一假设。现有基准多孤立研究单一因素,且集中于英语,难以反映真实场景下的模型表现。本文提出AVShift,首个针对德语的多分布偏移作者验证基准。该数据集包含超过15万对文本,覆盖三种体裁和21年跨度,支持在统一框架下控制评估跨体裁、时间与AI时代偏移的影响。我们评测了代表性基于特征、嵌入及大模型的方法。实验表明,微调后的大模型在跨体裁验证中泛化能力最强,且在风格多样的数据上训练收益显著。时间漂移是影响性能最强的因素,文档间时间间隔越大,准确率越低。相比之下,在AVShift中未发现可测量的AI时代风格漂移。特征分析揭示部分风格特征跨体裁保持稳定,但其重要性随体裁转换而变化。我们已公开数据集与代码,以促进后续研究。
原文摘要 · Abstract (English)
Authorship verification (AV) assumes that an author's writing style remains sufficiently stable to distinguish it from that of other writers. In practice, however, this assumption is challenged by distribution shifts caused by changes in genre, time, and AI-assisted writing. Existing AV benchmarks typically study these factors in isolation and focus predominantly on English, limiting our understanding of model robustness under realistic conditions. We introduce AVShift, the first German benchmark for systematically evaluating AV under multiple distribution shifts. AVShift comprises over 150,000 text pairs spanning three genres and 21 years, enabling controlled evaluation of cross-genre, temporal, and AI-era shifts within a unified framework. We benchmark representative feature-based, embedding-based, and LLM-based approaches. Our experiments show that fine-tuned LLMs generalize best across genres and benefit substantially from stylistically diverse training data. We further demonstrate that temporal drift is one of the strongest factors affecting AV, with performance degrading significantly as the time gap between documents increases. In contrast, we find no evidence of a measurable AI-era distribution shift within AVShift. Finally, our feature analysis reveals stylistic features that remain stable across genres, while their relative importance varies depending on the specific genre transition. We release AVShift and our code for future research.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。