OpenAI's Viral App ChatGPT Gets Two New Upgrades and a New Voice

Tuesday - September 26 2023, 17:09 UTC - 1 year ago

ChatGPT, OpenAI's viral chatbot app, has had two major updates; users can now interact with the chatbot through voice, and also ask it questions about images. There are five synthetic voices available to choose from, and the image recognition feature has been tried by a company called Be My Eyes. This set of upgrades adds further power and flexibility to the chatbot.

In one of the biggest updates to ChatGPT yet, OpenAI has launched two new ways to interact with its viral app.

First, ChatGPT now has a voice. Choose from one of five lifelike synthetic voices and you can have a conversation with the chatbot as if you were making a call, getting responses to your spoken questions in real time.

ChatGPT also now answers questions about images. OpenAI teased this feature in March with its reveal of GPT-4 (the model that powers ChatGPT), but it has not been available to the wider public before. This means that you can now upload images to the app and quiz it about what they show.

These updates join the announcement last week that DALL-E 3, the latest version of OpenAI's image-making model, will be hooked up to ChatGPT so that you can get the chatbot to generate pictures.The ability to talk to ChatGPT draws on two separate models. Whisper, OpenAI’s existing speech-to-text model, converts what you say into text, which is then fed to the chatbot. And a new text-to-speech model converts ChatGPT’s responses into spoken words.

In a demo the company gave me last week, Joanne Jang, a product manager, showed off ChatGPT’s range of synthetic voices. These were created by training the text-to-speech model on the voices of actors that OpenAI had hired. In the future it might even allow users to create their own voices. "In fashioning the voices, the number-one criterion was whether this is a voice you could listen to all day," she says.They are chatty and enthusiastic but won’t be to everyone’s taste. "I’ve got a really great feeling about us teaming up," says one. "I just want to share how thrilled I am to work with you, and I can’t wait to get started," says another. "What’s the game plan?" .

This grab bag of updates shows just how fast OpenAI is spinning its experimental models into desirable products. OpenAI has spent much of the time since its surprise hit with ChatGPT last November polishing its technology and selling it to both private consumers and commercial partners.

ChatGPT Plus, the company’s premium app, is now a slick one-stop shop for the best of OpenAI’s models, rolling GPT-4 and DALL-E into a single smartphone app that rivals Apple’s Siri, Google Assistant, and Amazon’s Alexa.

What was available only to certain software developers a year ago is now available to anyone for $20 a month. "We’re trying to make ChatGPT more useful and more helpful," says Jang.

In last week’s demo, Raul Puri, a scientist who works on GPT-4, gave me a quick tour of the image recognition feature. He uploaded a photo of a kid’s math homework, circled a Sudoku-like puzzle on the screen, and asked ChatGPT how you were meant to solve it. ChatGPT replied with the correct steps.

Puri says he has also used the feature to help him fix his fiancée’s computer by uploading screenshots of error messages and asking ChatGPT what he should do. "This was a very painful experience that it helped me get through," he says.

ChatGPT’s image recognition ability has already been trialed by a company called Be My Eyes, which makes an app for people with impaired vision. Users can upload a photo of what’s in front of them, such as a piece of mail, and the app uses facial recognition to identify the object.

This latest set of upgrades adds more power and flexibility to ChatGPT, making it just a little bit closer to a real-life artificial intelligence. "Everyone thought ChatGPT was impressive when it first came out," says Jang. "But what we’re doing now is so much bigger. " .