Soul Research

SoTA in voice cloning

Meet Soul TTS, leading voice cloning with human-like prosody.

Soul TTS architecture: a speaker encoder and a text encoder condition a diffusion transformer, and an audio VAE decodes the result Reference audioA recording from any room Text encoderBidirectional encoding with CTC Speaker encoderCausal encoding in patches Diffusion transformer Joint attentionQueries: frames; keys and values: frames, speaker, text AdaLN-Zero MLPConditioned on time and speaker Rectified flowRefines noise into clean audio latents for every frame Audio VAE decoderCausal decoding for live calls

Making AI FaceTime

Live video with human expression, reacting as you talk.

Hand holding a phone on a FaceTime call

Playing video games with you

Soul plays video games with you, just like a human friend.

Minecraft friends around a campfire in a snowy village at night, with Soul in the middle

Solving Karpathy's Memory Problem

Memory like a human’s, leading persona benchmarks.

Person speaking into a phone near a window