Hugging Face Daily Papers·· 29 days agoAI score38
RubatoCodec: Adaptive speech tokenization and streaming decoding with a budget
RubatoCodec: Budgeted Adaptive Speech Tokenization with Streaming Decoding
AI briefing
RubatoCodec is a variable-frame-rate speech codec based on X-Codec-2, achieving adaptive tokenization through utterance-level event budgets and streaming decoding. Its Adaptive Firing mechanism uses codec features and waveform spectral changes to divide variable-length events without needing a learned boundary predictor. At low event rates, reconstruction fidelity and intelligibility outperform fixed-frame-rate coding, intelligibility remains best on English and Japanese benchmarks, and in matched TTS comparisons WER is lower while UTMOS and UTMOSv2 are higher.
Open original arxiv.org