The launch of ChatGPT Search this week has reignited the battle for search supremacy, echoing the dynamics of the 1990s when AltaVista, Yahoo, and others competed for users’ attention. This time, the prize isn’t just about providing links – it’s about delivering direct answers to complex questions.
Google’s response in recent months with its AI-powered summaries reflects the tension at the heart of this shift. Whilst introducing search enhanced by Gemini the company is also trying to balance innovation against protecting its $200+ billion advertising business. Other players face no such constraints. Perplexity has gained 15 million monthly users by focusing on cited, accurate responses, although is toying with implementing advertising. More options: the existing competitor Bing, You.com which provides specialised tools for coding and content creation, and Phind which serves developers with complex answers, complete the current main choices.
We tested them with a complex query on the very recent UK Autumn budget and comparative AI-linked fiscal policy in other countries. Claude analysed the results over several repetitions (as these tools tend to respond differently each query):
- Perplexity: Strong on recent data and specific figures with good citations but did notably fail to address a specific part of the query. Score: 84% (Paid version)
- ChatGPT Search: Showed good judgment by correctly noting absence of specific AI measures in the budget and provided balanced international comparison with specific figures. Score: 70% (Paid version)
- You.com: Attempted comprehensive coverage with clear structure but mixed current/speculative content and included unnecessary images while lacking clear citations. Score: 62% (Free version)
- Phind: Made assumptions about future events and lacked specific data but provided good structural analysis – shows risks of AI search going beyond verified facts. Score: 48% (Free version)
- Bing: Provided a well-structured and detailed analysis with plausible content and figures, but critically undermined its credibility by fabricating future sources and dates. Score: 46% (Paid Copilot version)
We also tested the tools on a slightly less high-brow topic, the most recent Strictly Come Dancing contestant leaderboard. ChatGPT Search won hands-down as it managed to source a web page with detailed results, something the others didn’t find. This highlights one of the main challenges with AI search… if the results found are good, the output tends to be better. But often the links retrieved feel quite random, lacking the relevance that we expect from Google.
One of the main advantages of AI search is being able to continue the conversation. Once the first query is complete these tools allow follow-up questions to be asked which can be incredibly powerful. Perplexity goes further and allows those questions to be laid out in an information page you can re-order and share.
Our final testing looked at this week’s new search or ‘grounding’ capabilities in Google Gemini 1.5 Pro. It seems that this may be useful for specific factual queries but was limited to only a few search results with little control over what they were. Whilst Google still have by far the most powerful web search capability, that is still not integrated into their Gemini products flexibly. At ExoBrain we’ll continue to use third party search to feed our ‘grounded’ AI solutions for the time being.
Takeaways: Different search tools serve different purposes. Despite the launch of ChatGPT Search, Perplexity still feels like the leader, with the ability to focus on academic, social or web source content, the power of o1-preview for advanced analysis or the user’s choice of frontier model, and its publishing tools. The overall ChatGPT experience will benefit from search with clear cited content items that are returned almost instantly. Google has strong models and the world’s best web search, but the innovators dilemma seems to be preventing it from integrating these assets to full effect.
We’d summarise our findings as follows:
- Use web search when you need verified, current facts or specific pages you know are out there – it’s about finding rather than understanding.
- Use AI search when you need to understand broad patterns or relationships across multiple current sources and continue the conversation – it’s about synthesis of current events and dynamic topics rather than 1-off high-precision facts.
- Use non-search AI when you need analysis of established knowledge or creative thinking – it’s about focused thought rather than being distracted by web info which can often be of lower quality than the concepts already in the model.