The Block’s latest AI model Dessa was trained entirely on human speech – and it shows

If interacting with Siri is the extent of your experience of “talking” to an AI, you’re in for a rude awakening. While Apple’s voice assistant often botches basic commands and can take an unnaturally long time to process a request, the latest and greatest AI models are growing closer and closer to natural human speech.

In this case: Dessa, the brainchild of Jack Dorsey’s Block, It is reportedly the world’s first full-duplex speech modelmeaning it can hear you And Speak in unison, just like a real human being would during a normal conversation. In fact, this is the charm of DESSA: it was trained entirely on human speech to make it sound as natural-sounding as possible.

This tweet is currently unavailable. It may be loading or may have been removed.

But these claims must be made heard to be appreciated, and Block Press Release Dessa’s announcement shows how far AI models have come in predicting the rhythms of natural human conversation.

In Second sample audio clipFor example, a real-life human and Dessa have a lengthy conversation about Christopher Nolan’s films, and although the model expands its knowledge at some points, providing the title of a film for example, it does so in a natural rather than robotic manner that, to human ears, mirrors the tone of its human interlocutor, in keeping with a perfectly normal back-and-forth about Nolan’s cinematic world.

In the third audio sample, Dessa spontaneously reacts with genuine surprise and excitement at an unexpected cue from a human (“Are you excited for our anniversary?”), with conversation markers you’d expect from a human: audible gasps, a joyous excitement that turns to tears, and even difficulty finding the right words.

The final clip, which completely eliminates human involvement, consists of a conversation between ChatGPT and Dessa, and the contrast between the ChatGPT agent’s robotic, unnatural speech patterns and Dessa’s prompts, interruptions, and puzzled questions perfectly demonstrates how far Block has advanced its AI speech model.

Perhaps most impressively, Dessa understands when a human is being sarcastic when talking to her, which requires a sophisticated understanding of both tone and its relationship to the content of speech, which adult humans easily process, but that has historically stymied AI models (don’t even think about talking to Siri in a sarcastic tone).

According to the same press release, Block’s motivation for developing DESSA is simple:

“Speech is the most natural interface to any intelligence, super or otherwise, and today’s voice AI ruins it. It flattens what we say into text and can’t keep up with human time. AI keeps gaining power and independence, and it’s still nearly deaf to the emotions in our voices. Imagine being in severe pain when a voice AI has no idea and just keeps going on and on. That’s a security problem. DESSA is one step toward fixing this. It’s a step.”

Leave aside the question of whether it is the same to recognize a tone of urgency, fear, or pain in a voice pattern. Sympathy With the feelings behind those markers. But one must take a second to appreciate the complexity of training agent AI to understand them in the first place.

Wherever it takes us, DESSA is undoubtedly a major advancement in artificial intelligence and, more importantly, in bridging the gap between human communication and AI understanding.