构建首个跨LLM时代的科学摘要多粒度基准，揭示写作方式变革。
SciZoom: A Large-scale Benchmark for Hierarchical Scientific Summarization across the LLM Era
- 按时间分层设计，覆盖2020-2025年4万篇顶会论文
- 支持摘要、贡献点、TL;DR三层次压缩，最高达600:1
- 发现LLM写作使表达更自信但趋同，公式化表达增10倍
AI研究的爆炸式增长带来了前所未有的信息过载，推动对多层次科学摘要的需求。尽管大模型（LLMs）被广泛用于摘要生成，现有基准在规模、粒度多样性及时代代表性上仍显不足，且大多发布于2022年11月前。自ChatGPT发布以来，研究人员迅速采用大模型撰写论文，深刻改变了科研写作范式，但缺乏分析其演变的资源。为此，我们提出SciZoom，一个包含44,946篇来自NeurIPS、ICLR、ICML、EMNLP四大会刊（2020–2025年）的基准数据集，明确划分为预大模型与后大模型时期。SciZoom提供三个层级摘要目标（摘要、贡献、TL;DR），实现高达600:1的压缩比，支持多粒度摘要研究与科学写作演化分析。语言学分析显示，句式模式发生显著变化（公式化表达最高提升10倍），修辞风格趋于确定性（模糊表达下降23%），表明大模型辅助写作催生更自信但趋同的文风。SciZoom既是挑战性基准，也是探索生成式AI时代科学话语演化的独特资源。代码与数据集已开源至GitHub与Hugging Face。
原文摘要 · Abstract (English)
The explosive growth of AI research has created unprecedented information overload, increasing the demand for scientific summarization at multiple levels of granularity beyond traditional abstracts. While LLMs are increasingly adopted for summarization, existing benchmarks remain limited in scale, target only a single granularity, and predate the LLM era. Moreover, since the release of ChatGPT in November 2022, researchers have rapidly adopted LLMs for drafting manuscripts themselves, fundamentally transforming scientific writing, yet no resource exists to analyze how this writing has evolved. To bridge these gaps, we introduce SciZoom, a benchmark comprising 44,946 papers from four top-tier ML venues (NeurIPS, ICLR, ICML, EMNLP) spanning 2020 to 2025, explicitly stratified into Pre-LLM and Post-LLM eras. SciZoom provides three hierarchical summarization targets (Abstract, Contributions, and TL;DR) achieving compression ratios up to 600:1, enabling both multi-granularity summarization research and temporal mining of scientific writing patterns. Our linguistic analysis reveals striking shifts in phrase patterns (up to 10x for formulaic expressions) and rhetorical style (23% decline in hedging), suggesting that LLM-assisted writing produces more confident yet homogenized prose. SciZoom serves as both a challenging benchmark and a unique resource for mining the evolution of scientific discourse in the generative AI era. Our code and dataset are publicly available on GitHub (https://github.com/janghana/SciZoom) and Hugging Face (https://huggingface.co/datasets/hanjang/SciZoom), respectively.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。