A Survey of Quality Evaluation on AI-Generated Lyrics

Abstract

With the rapid development of generative artificial intelligence, AI lyric generation tools such as Suno, Baidu Wenxin Lyric Generator, and Doubao have been widely deployed. However, AI-generated lyrics commonly suffer from thematic deviation, emotional fragmentation, stylistic mismatch, grammatical inconsistency, and long-structure logical collapse, making objective quality evaluation a critical link for optimizing generation models and filtering outputs. This paper systematically reviews existing research on AI-generated lyric quality evaluation along three main lines: evaluation dimensions, datasets, and technical routes. Existing approaches are categorized into four technical routes: manual subjective evaluation, traditional statistical and rule-based modeling, deep learning and representation learning, and large language model and multimodal intelligent evaluation. A three-layer progressive evaluation indicator system is constructed, comprising the textual language layer, lyric-music layer, and artistic aesthetic layer. The findings reveal four key bottlenecks: the deep gap between automatic quantitative metrics and human subjective aesthetic experience, the modeling blind spot for Chinese phonological features and lyric-music coordination, the severe scarcity of dedicated annotated datasets with uneven cultural distribution, and the disconnect between static text quality evaluation and dynamic human-computer collaboration experience. Future research should advance in three directions: developing lyric-specific vertical large models integrating music theory, constructing audio-text cross-modal joint lyric-music evaluation systems, and building generation-evaluation-optimization closed-loop ecosystems for the entire creative workflow.