arXiv:2607.08526cs.SDcs.PF2026-07

轻量级音频生成框架,让手机级设备也能实时创作带语义控制的音乐。

A Quantized Native Runtime for On-Device Semantic Audio Generation

  • 全原生运行,无需依赖库,直接在普通CPU/GPU和树莓派上执行文本转音乐。
  • 8位量化几乎无质量损失,4位量化使12亿参数模型可在8GB树莓派运行。
  • 内置生成控制机制,可精准调节音乐风格与情感特征,适合边缘设备应用。

语义音频应用正转向在消费级和嵌入式硬件上实现可控生成,而非依赖重载的数据中心框架。我们提出aria,一个无依赖的原生运行时,能在普通GPU、纯CPU设备及树莓派5上完整运行Stable Audio 3(SA3)的文本到音乐全流程,无需Python或深度学习框架支持。核心贡献在于量化研究:通过降低数值精度适配严苛内存限制,以节省内存而非增加内存。由于运行时掌控所有内部张量,还引入了激活控制(activation steering),一种低成本的生成内容调控方式。通过三项独立指标(提示遵循度、整体音质、风格保留性)评估质量损失,均以随机种子间差异为基准。8位精度在所有指标上均无显著质量下降,且在GPU上速度最快;4位精度带来微小但有界的代价,却足以将12亿参数模型部署于8GB树莓派。相比官方实现,aria在生成速度上持平或超越,启动时间快约7倍。对控制接口的案例研究成功生成具有味觉联想(‘声学调味’)的音乐,实现对部分属性的真实但有限的控制。这些结果表明,紧凑且量化的原生运行时是物联网声音场景中设备端语义音频的可行基础。aria代码已开源:https://github.com/matteospanio/aria。

原文摘要 · Abstract (English)

Semantic audio applications increasingly require controllable generation on commodity and embedded hardware rather than through framework-heavy datacenter stacks. We present \textit{aria}, a dependency-free native runtime that runs the complete text-to-music pipeline of Stable Audio~3 (SA3) on ordinary GPUs, CPU-only machines, and a Raspberry~Pi~5, with no Python or deep-learning framework underneath. Our main contribution is a study of quantization: running the model at lower numerical precision to fit tight memory budgets, saving memory in place rather than adding to it. Because the runtime owns every internal tensor, it also exposes activation steering, a low-cost way to steer what the model generates. We judge the quality cost with three independent measures of the output (prompt adherence, overall audio quality, taste preservation), each compared against the ordinary variation between random seeds. Eight-bit precision shows no measurable quality loss on any measure while sharply cutting memory, and it is the fastest mode on the GPU; four-bit adds a small, bounded cost but shrinks the footprint enough to run the $1.2$-billion-parameter model on an $8$\,GB Pi. Against the official implementation, aria matches or exceeds generation speed and starts about seven times faster. A case study of the steering interface generates music carrying taste associations (\emph{sonic seasoning}), with genuine but bounded control for a subset of attributes. These results make a compact, quantized runtime with built-in control a practical basis for on-device semantic audio in Internet-of-Sounds settings. The \textit{aria} runtime is released at https://github.com/matteospanio/aria.

音频生成量化边缘计算树莓派

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。