The Race to Build a ChatGPT-Powered Search Engine

Another issue with a system like ChatGPT is that its responses a based on only the data it was trained on. Retraining the model in its entirety can cost millions of dollars because of its size and the scale of the data. You Chat is confused when asked for the latest sports scores but knows what the weather is like in New York at the moment. Socher doesn’t want to disclose how up-to-date information is incorporated, seeing it as a competitive advantage.

“I think right now a lot of these chat interfaces are way superior to the search experience in some ways, but in others they’re clearly still much worse,” Socher says. “We’re working on reducing all these issues.”

Aravind Srinivas, founder and CEO of search startup Perplexity AI, who previously worked at OpenAI, says the challenge of updating a ChatGPT-like system with recent information means that they need to be combined with something else. “Alone they’ll never be able to be good search engines,” he says.

Saam Motamedi, a venture capitalist at Greylock Partners who has invested in the AI-based search company Neeva, says it is also unclear how compatible chat interfaces are with the primary revenue model for search engines—advertising. Google and Bing use search queries to select ads that appear on top of the list of links served up in response. Motamedi suspects that new forms of advertising might need to emerge for chat-style search interfaces to be viable, but it isn’t altogether clear what those will be. Neeva charges a subscription fee for unlimited ad-free searches.

The cost of running a model like ChatGPT on the scale of Google might also prove problematic. Luis Ceze, cofounder and CEO of OctoML, a company that helps companies lower the cost of deploying machine learning algorithms, estimates that it may be 10 times more expensive to run a ChatGPT search than a Google search, because each answer requires running a large and complex AI model.

The scale of ChatGPT mania has taken some coders and AI researchers familiar with the underlying technology by surprise. The algorithm at the core of the bot, called GPT, was first developed by OpenAI in 2018, and a more powerful version, GPT-2, was revealed in 2019. It is a machine learning model designed to take in text and then predict what comes next, which OpenAI showed can perform impressively if trained with huge volumes of text. The first commercial version of the technology, GPT-3, has been available for developers to use since June 2020 and can accomplish many of the things ChatGPT has recently been feted for.

ChatGPT uses an improved version of the underlying algorithm, but the biggest leap in its abilities comes from OpenAI having humans provide feedback to the system on what makes a satisfying answer. But like the text-generation systems before it, ChatGPT is still prone to reproducing biases from its training data as well as “hallucinating” plausible but incorrect results.

Gary Marcus, a professor emeritus at New York University and a vocal critic of AI hype, believes ChatGPT is unsuited to search because it has no true understanding of what it says. He adds that tools like ChatGPT may cause other problems for search companies by flooding the internet with AI-generated, search engine-optimized text. “All search engines are about to have a problem,” he says.

Alex Ratner, an assistant professor at the University of Washington and cofounder of Snorkel AI, which works on training AI models more efficiently, calls ChatGPT “legitimately an inflection” in what software can do. But he also says that it may take a while to figure out how to prevent language models like GPT from making things up. He believes that finding a way to keep them up to date with new information to keep search fresh will most likely involve new approaches to training the underlying AI models.

How long those fixes will take to invent and prove out is unclear. It may be some time before the technology can radically change the way people search for answers, even if other use cases come to pass, such as dreaming up new recipes or serving as a study or programming buddy. “It’s amazing, and I told my team that people are going to see years as pre- and post-ChatGPT,” says Chen of Moveworks. “But whether it will replace search is a different question.”