音乐情绪能精准操控图像风格,让视觉更打动人心
Aesthetic Matters in Music Perception for Image Stylization: A Emotion-driven Music-to-Visual Manipulation
- 用音乐的音高节奏提取情绪,反向控制图像色彩光影
- 实测显示生成图像在美学评分和脑电反应上显著提升
- 适合创意设计、交互艺术等需要情感共鸣的场景
情绪信息对增强人机交互和深化图像理解至关重要。尽管深度学习推动了图像识别的发展,但图像中情感表达的直观理解和精确控制仍具挑战。类似地,音乐研究多聚焦理论,对其情绪维度及与视觉艺术的融合探索有限。为此,我们提出 EmoMV——一种基于情绪驱动的音乐到视觉调控方法,通过音乐元素(如音高、节奏)的底层分析,结合上层情绪映射,实现对图像颜色、光照等视觉属性的精准调控。采用多尺度评估框架,包含图像质量指标、美学评价及实时脑电(EEG)测量,以捕捉用户情感响应。结果表明,EmoMV 能有效将音乐情绪内容转化为具有视觉冲击力的图像,推动跨模态情感融合,为创意产业与交互技术开辟新路径。
原文摘要 · Abstract (English)
Emotional information is essential for enhancing human-computer interaction and deepening image understanding. However, while deep learning has advanced image recognition, the intuitive understanding and precise control of emotional expression in images remain challenging. Similarly, music research largely focuses on theoretical aspects, with limited exploration of its emotional dimensions and their integration with visual arts. To address these gaps, we introduce EmoMV, an emotion-driven music-to-visual manipulation method that manipulates images based on musical emotions. EmoMV combines bottom-up processing of music elements-such as pitch and rhythm-with top-down application of these emotions to visual aspects like color and lighting. We evaluate EmoMV using a multi-scale framework that includes image quality metrics, aesthetic assessments, and EEG measurements to capture real-time emotional responses. Our results demonstrate that EmoMV effectively translates music's emotional content into visually compelling images, advancing multimodal emotional integration and opening new avenues for creative industries and interactive technologies.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。