SoundStorm AI-Based SoundStorm for Rapid Audio Generation

SoundStorm is a ground-breaking AI model developed by Google Research for efficient and non-autoregressive audio generation. It operates by taking the semantic tokens of AudioLM as input, employing bidirectional attention and confidence-based parallel decoding to produce the tokens of a neural audio codec. This unique approach allows SoundStorm to generate high-quality audio with remarkable speed and consistency, outperforming the autoregressive generation approach of AudioLM in terms of speed and voice consistency. The model can create 30 seconds of audio in just half a second on a TPU-v4, which is two orders of magnitude faster than conventional methods. Moreover, it is capable of scaling audio generation to longer sequences, synthesizing high-quality, natural dialogue segments from a given transcript annotated with speaker turns and a short prompt with the speakers’ voices.