ChatGPT has been updated with support for voice conversations and image recognition, OpenAI announced Monday. The company’s AI-powered chatbot will soon be able to understand images captured or shared by users and provide details or related information on platforms where the chatbot is available. It will also be able to talk back and forth using OpenAI’s Whisper speech recognition tool and a new text-to-speech (TTS) technology from the company that it claims will offer “human-like” audio on the ChatGPT app from the company. smartphones.
OpenAI revealed in a blog post that the company’s new image recognition feature for ChatGPT will be available on all platforms, while the voice calling feature will be available on iOS and Android via an opt-in setting. These features will be available to ChatGPT Plus and Enterprise subscribers, and there’s no word on whether it will be rolled out to users on the free tier in the future.
The voice calls coming to ChatGPT can be enabled by going to Institutions > New functions and toggle the option to enable voice calling. You can then choose from five voices. OpenAI says it has worked with professional voice actors to offer the new feature. The ChatGPT app can answer questions by converting your spoken questions into text that can be understood by the chatbot, and answers are converted into audio using the company’s new TTS technology.
ChatGPT isn’t the only service that will use OpenAI’s new TTS technology – Spotify on Monday announced a new AI-based voice translation tool for podcast creators that can automatically translate a podcast from English to French, German and Spanish. The tool is being tested with a few podcast hosts and translated episodes will be available to all users where Spotify is available, the streaming platform said.
OpenAI says the new image recognition tool runs on the company’s multimodal GPT-3.5 and GPT-4 models and is capable of analyzing images and text in photos, screenshots and documents. Users can capture an image or share an existing image on their phone with ChatGPT to get insights from the chatbot.
ChatGPT also allows users to share multiple images that can be discussed with the chatbot, according to OpenAI. If you want the focus to be on a specific area, you can highlight part of the image using the built-in drawing tool. For example, if you draw a loose bicycle chain in a photo shared with ChatGPT, the chatbot can show you ways to fix the problem.
