用AI生成代码模式，自动检测科研代码中的方法学错误。
scicode-lint: Detecting Methodology Bugs in Scientific Python Code with LLM-Generated Patterns
- 用大模型生成检测模式，而非手动编码
- 在真实科研代码中实现62%精度的错误检测
- 适合科研人员和代码审核者快速发现隐藏漏洞
科学计算中的方法学错误会导致看似合理却错误的结果，传统静态分析工具难以捕捉。尽管已有研究构建了针对机器学习的检查工具，但存在依赖特定pylint版本、打包困难、需人工设计每种模式等可持续性问题。随着AI生成代码增多，自动化方法学检测需求上升。我们提出scicode-lint，采用两层架构：构建时使用前沿大模型生成检测模式，运行时由轻量本地模型执行。模式由生成而非手写，适应新库版本仅需消耗少量令牌，无需工程投入。在标注真实的Kaggle笔记本上，预处理泄漏检测达到65%精确率、100%召回率；在38篇发表论文中，经大模型评估的精确率为62%，不同模式间差异显著；在预留测试集上精确率为54%。控制实验中，对66种模式整体准确率达97.7%。
原文摘要 · Abstract (English)
Methodology bugs in scientific Python code produce plausible but incorrect results that traditional linters and static analysis tools cannot detect. Several research groups have built ML-specific linters, demonstrating that detection is feasible. Yet these tools share a sustainability problem: dependency on specific pylint or Python versions, limited packaging, and reliance on manual engineering for every new pattern. As AI-generated code increases the volume of scientific software, the need for automated methodology checking (such as detecting data leakage, incorrect cross-validation, and missing random seeds) grows. We present scicode-lint, whose two-tier architecture separates pattern design (frontier models at build time) from execution (small local model at runtime). Patterns are generated, not hand-coded; adapting to new library versions costs tokens, not engineering hours. On Kaggle notebooks with human-labeled ground truth, preprocessing leakage detection reaches 65% precision at 100% recall; on 38 published scientific papers applying AI/ML, precision is 62% (LLM-judged) with substantial variation across pattern categories; on a held-out paper set, precision is 54%. On controlled tests, scicode-lint achieves 97.7% accuracy across 66 patterns.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。