用强化学习优化代码水印的令牌选择，让水印隐蔽且可检测。
Optimizing Token Choice for Code Watermarking: An RL Approach
- 通过策略模型智能调整下一个令牌的选择，嵌入水印。
- 在保持代码功能的同时，水印检测率显著高于现有方法。
- 适合关注大模型代码版权保护的研究者和开发者。
保护大语言模型生成代码的知识产权需要有效的水印系统，但代码具有高度结构化和语法约束的特点。本文提出 CodeTracer，一种基于强化学习的自适应代码水印框架。其核心是策略驱动的方法，利用参数化模型在下一个令牌预测中智能偏置令牌选择，使嵌入的水印在保持代码功能的同时，呈现细微但统计上可检测的分布偏差。为支持策略学习，设计了融合执行反馈与水印信号的综合奖励机制，平衡过程级与结果级奖励。同时采用 Gumbel Top-k 重参数化，实现离散水印决策的梯度优化。大量对比实验表明，CodeTracer 在水印可检测性和代码功能保留方面均显著优于当前最优基线。
原文摘要 · Abstract (English)
Protecting intellectual property on LLM-generated code necessitates effective watermarking systems that can operate within code's highly structured, syntactically constrained nature. In this work, we introduce CodeTracer, an innovative adaptive code watermarking framework underpinned by a novel reinforcement learning training paradigm. At its core, CodeTracer features a policy-driven approach that utilizes a parameterized model to intelligently bias token choices during next-token prediction. This strategy ensures that embedded watermarks maintain code functionality while exhibiting subtle yet statistically detectable deviations from typical token distributions. To facilitate policy learning, we devise a comprehensive reward system that seamlessly integrates execution feedback with watermark embedding signals, balancing process-level and outcome-level rewards. Additionally, we employ Gumbel Top-k reparameterization to enable gradient-based optimization of discrete watermarking decisions. Extensive comparative evaluations demonstrate CodeTracer's significant superiority over state-of-the-art baselines in both watermark detectability and the preservation of generated code's functionality. Our code is available at https://github.com/TimeLovercc/CodeTracer.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。