Skip to content
Hugging Face Daily Papers·· 29 days agoAI score38

RubatoCodec: Adaptive speech tokenization and streaming decoding with a budget

RubatoCodec: Budgeted Adaptive Speech Tokenization with Streaming Decoding

AI briefing

RubatoCodec is a variable-frame-rate speech codec based on X-Codec-2, achieving adaptive tokenization through utterance-level event budgets and streaming decoding. Its Adaptive Firing mechanism uses codec features and waveform spectral changes to divide variable-length events without needing a learned boundary predictor. At low event rates, reconstruction fidelity and intelligibility outperform fixed-frame-rate coding, intelligibility remains best on English and Japanese benchmarks, and in matched TTS comparisons WER is lower while UTMOS and UTMOSv2 are higher.

Open original arxiv.org