Stability AI optimized its audio generation model to run on Arm chips

AI startup Stability AI has teamed up with chipmaker Arm to bring Stability’s Stable Audio Open, an AI model that can generate audio including sound effects, to mobile devices running Arm chips.

While a number of AI-powered apps can generate audio, like Suno and Udio, most rely on cloud processing, meaning that they can’t be used offline. Moreover, some audio generation models were trained on copyrighted content — posing an IP risk. Stability claims Stable Audio Open’s training set is made up entirely of royalty-free audio and songs.

Stable Audio Open running on Arm chips, which will be demoed at the Mobile World Congress conference in Barcelona this week, can generate a sound from a text description like, “Gentle ocean waves at sunset.” Stability says that it worked with Arm to optimize and “distill” Stable Audio Open, speeding up generation times by 30x. Generating a single 11-second audio sample takes around 8 seconds on an Armv9 CPU.

To be clear, the optimized Stable Audio Open model isn’t available to download — at least not yet. But in a statement, Stability CEO Prem Akkaraju hinted that Stability will work to bring its models, including Stable Audio Open, to consumer apps and devices in the future.

“As more and more professional creatives and businesses adopt generative AI to power their production pipeline, it’s important that our models and workflows are available everywhere for builders to build and creators to create,” Akkaraju said. “We are excited to partner with Arm for this exact reason.”

Stability says that it’s collaborating with Arm to further optimize Stable Audio Open.

Stability, the beleaguered firm behind the popular image generation model Stable Diffusion, raised new cash last year as investors including Eric Schmidt and Napster founder Sean Parker sought to turn the business around. Emad Mostaque, Stability’s co-founder and ex-CEO, reportedly mismanaged Stability into financial ruin, leading staff to resign, a partnership with Canva to fall through, and investors to grow concerned about the company’s prospects.

In the last few months, Stability has hired a new CEO, appointed Titanic director James Cameron to its board of directors, and released several new image generation models.