Authors
- Michele Mancusi
- Yurii Halychanskyi
- Kin Wai Cheuk
- Eloi Moliner
- Chieh-Hsin Lai
- Stefan Uhlich*
- Junghyun Koo*
- Marco A. Martínez-Ramírez
- Wei-Hsiang Liao
- Giorgio Fabbro*
- Yuki Mitsufuji
* External authors
Venue
- ICASSP-25
Date
- 2025
Latent Diffusion Bridges for Unsupervised Musical Audio Timbre Transfer
Michele Mancusi
Yurii Halychanskyi
Kin Wai Cheuk
Eloi Moliner
Stefan Uhlich*
Marco A. Martínez-Ramírez
Giorgio Fabbro*
* External authors
ICASSP-25
2025
Abstract
Music timbre transfer is a challenging task that involves modifying the timbral characteristics of an audio signal while preserving its melodic structure. In this paper, we propose a novel method based on dual diffusion bridges, trained using the CocoChorales Dataset, which consists of unpaired monophonic single-instrument audio data. Each diffusion model is trained on a specific instrument with a Gaussian prior. During inference, a model is designated as the source model to map the input audio to its corresponding Gaussian prior, and another model is designated as the target model to reconstruct the target audio from this Gaussian prior, thereby facilitating timbre transfer. We compare our approach against existing unsupervised timbre transfer models such as VAEGAN and Gaussian Flow Bridges (GFB). Experimental results demonstrate that our method achieves both better Fréchet Audio Distance (FAD) and melody preservation, as reflected by lower pitch distances (DPD) compared to VAEGAN and GFB. Additionally, we discover that the noise level from the Gaussian prior, σ, can be adjusted to control the degree of melody preservation and amount of timbre transferred.
Related Publications
Consistency Training (CT) has recently emerged as a promising alternative to diffusion models, achieving competitive performance in image generation tasks. However, non-distillation consistency training often suffers from high variance and instability, and analyzing and impr…
Detecting musical versions (different renditions of the same piece) is a challenging task with important applications. Because of the ground truth nature, existing approaches match musical versions at the track level (e.g., whole song). However, most applications require to …
Diffusion models have demonstrated exceptional performances in various fields of generative modeling, but suffer from slow sampling speed due to their iterative nature. While this issue is being addressed in continuous domains, discrete diffusion models face unique challenge…
JOIN US
Shape the Future of AI with Sony AI
We want to hear from those of you who have a strong desire
to shape the future of AI.