Authors
- Yunkee Chae
- Woosung Choi
- Yuhta Takida
- Junghyun Koo*
- Yukara Ikemiya
- Zhi Zhong*
- Kin Wai Cheuk
- Marco A. Martínez-Ramírez
- Kyogu Lee*
- Wei-Hsiang Liao
- Yuki Mitsufuji
* External authors
Venue
- ICASSP-25
Date
- 2025
VRVQ: Variable Bitrate Residual Vector Quantization for Audio Compression
Yunkee Chae
Woosung Choi
Yukara Ikemiya
Zhi Zhong*
Kin Wai Cheuk
Marco A. Martínez-Ramírez
Kyogu Lee*
* External authors
ICASSP-25
2025
Abstract
Recent state-of-the-art neural audio compression models have progressively adopted residual vector quantization (RVQ). Despite this success, these models employ a fixed number of codebooks per frame, which can be suboptimal in terms of rate-distortion tradeoff, particularly in scenarios with simple input audio, such as silence. To address this limitation, we propose variable bitrate RVQ (VRVQ) for audio codecs, which allows for more efficient coding by adapting the number of codebooks used per frame. Furthermore, we propose a gradient estimation method for the non-differentiable masking operation that transforms from the importance map to the binary importance mask, improving model training via a straight-through estimator. We demonstrate that the proposed training framework achieves superior results compared to the baseline method and shows further improvement when applied to the current state-of-the-art codec.
Related Publications
We present Music Arena, an open platform for scalable human preference evaluation of text-to-music (TTM) models. Soliciting human preferences via listening studies is the gold standard for evaluation in TTM, but these studies are expensive to conduct and difficult to compare…
This paper explores the use of unlearning methods for training data attribution (TDA) in music generative models trained on large-scale datasets. TDA aims to identify which specific training data points contributed to the generation of a particular output from a specific mod…
Blind inverse problems, where both the target data and forward operator are unknown, are crucial to many computer vision applications. Existing methods often depend on restrictive assumptions such as additional training, operator linearity, or narrow image distributions, thu…
JOIN US
Shape the Future of AI with Sony AI
We want to hear from those of you who have a strong desire
to shape the future of AI.