DialBGM: A Benchmark for Background Music Recommendation from Everyday Multi-Turn Dialogues
A new benchmark reveals that today's best AI models can't pick fitting background music for a conversation — and even humans find it surprisingly hard.
The Thesis
Choosing background music that fits a conversation without drawing attention to itself turns out to be a hard AI problem nobody has formally studied before. DialBGM is the first benchmark designed to measure exactly that: given a multi-turn chat between two people, pick the track that sounds right without any explicit music cues in the text. The researchers tested more than a dozen models — including large multimodal language models and audio-language systems — and none cleared 35% accuracy on the top pick, compared to human agreement rates that are meaningfully higher. The catch is that this is a benchmark paper, not a solution: it diagnoses a gap but does not close it. The real-world stakes are real, though — automatic BGM selection matters for podcasts, video games, AI companions, and social media content tools.
Catalyst
Multimodal models capable of jointly reasoning about text and audio have only become widely available in the past two years, making it feasible to test whether off-the-shelf systems can bridge conversational context and musical mood. At the same time, the explosion of AI-assisted content creation — short video, AI podcasts, interactive agents — has created commercial pressure to automate BGM decisions that previously required a human music supervisor. Without a standardized benchmark, progress in this niche was unmeasurable; DialBGM provides the ruler.
What's New
Prior music recommendation research focused on explicit signals: user listening history, genre tags, lyric sentiment, or visual content from video frames. Those systems assume the input contains direct music descriptors like mood words or genre labels. DialBGM shifts the problem to everyday dialogues that contain no such cues — 'Did you call the plumber yet?' is representative — forcing models to infer emotional and social context from pure conversational dynamics. The paper also introduces multi-criteria human preference rankings (contextual relevance, non-intrusiveness, and consistency) as ground truth, rather than simpler binary relevance labels used in earlier retrieval benchmarks.
The Counter
A benchmark with 1,200 dialogues is a starting point, not a definitive statement about the problem's difficulty. Human inter-annotator agreement on 'appropriate background music' for a mundane conversation is likely low to begin with — people disagree sharply about whether silence, jazz, or lo-fi beats fit the same conversation. If the human ceiling itself is uncertain, the claim that models 'fall far short of human judgment' is hard to interpret. The paper also does not demonstrate that anyone actually wants automatic BGM for text conversations — the use case is clearest for video or podcast production, where visual and prosodic cues are available and deliberately excluded here. Existing production workflows already use human music supervisors or simple rule-based mood tagging, which may be good enough for most commercial applications. Finally, a benchmark that no model beats at 35% accuracy could mean the task is meaningfully hard — or it could mean the annotation methodology or candidate set design is flawed.
Longs
- SPOT (Spotify) — owns music licensing and AI recommendation infrastructure relevant to automated BGM
- META — short-video and Reels tooling benefits from automated background audio selection
- SOUN (SoundHound AI) — audio-language model developer with direct exposure to conversational audio products
- MSFT — Azure OpenAI and Copilot ecosystem; multimodal model provider tested in benchmarks like this
Shorts
- Manual music supervisors and BGM licensing services — automation of their core selection judgment, if models improve
- Existing music recommendation engines (Pandora/SiriusXM) — built on listening history, not conversational context; would need architectural rethink to compete in this space
Enablers (Picks & Shovels)
- CLAP (Contrastive Language-Audio Pretraining) — open-source audio-language embedding model that several benchmark baselines rely on
- Hugging Face audio model hub — hosts the open-source audio-language models evaluated in the paper
- MusicCaps and similar open audio datasets — foundational training data for audio understanding models
- LLM APIs (OpenAI, Anthropic, Google) — proprietary multimodal models tested as baselines
Private Watchlist
- Epidemic Sound — licensed music library provider building AI-assisted content tools
- Loudly — AI music generation and recommendation startup
- Musicbed — licensed music for video creators, potential automated curation use case
- ElevenLabs — audio AI platform expanding into ambient and generative audio
Resources
The Paper
Selecting an appropriate background music (BGM) that supports natural human conversation is a common production step in media and interactive systems. In this paper, we introduce dialogue-conditioned BGM recommendation, where a model should select non-intrusive, fitting music for a multi-turn conversation that often contains no music descriptors. To study this novel problem, we present DialBGM, a benchmark of 1,200 open-domain daily dialogues, each paired with four candidate music clips and annotated with human preference rankings. Rankings are determined by background suitability criteria, including contextual relevance, non-intrusiveness, and consistency. We evaluate a wide range of open-source and proprietary models, including audio-language models and multimodal LLMs, and show that current models fall far short of human judgments; no model exceeds 35% Hit@1 when selecting the top-ranked clip. DialBGM provides a standardized benchmark for developing discourse-aware methods for BGM selection and for evaluating both retrieval-based and generative models.