I wasn't surprised...
 |
| "Deep Research" as imagined by Google Gemini. I especially like the researcher writing simultaneously with both hands. I can't do that, can you? |
... when "Deep Research" suddenly became the thing all the cool AI kids were talking about. It's the inevitable next step in "Search" when you live in a world full of LLMs going their thing.
What is Deep Research?
You know how regular web search works. The whole premise of SearchResearch has been to figure out how to do deeper, more insightful research by using the search tools at hand.
"Deep Research" uses an LLM to do some of that work for us. Is it the next generation of SRS tools?
In effect, "Deep Research" is AI-guided search + LLM analysis to make multi-step investigations. The goal is to do some of the SearchResearch heavy lifting for you.
The key new idea of "Deep Research" is that the LLM AI uses its "reasoning model" to create a multi-step plan that it then executes to find, consolidate, and analyze information for you. This actually works pretty well because the DR system breaks the large research task down into multiple steps, does a bunch of searches to find relevant documents, then answers the question in the context of those retrieved documents. It's fairly clever.
You'll probably read that this is one of the first uses of "Agent AI" (aka "agentic"), but it's kinda not really that. I think most people agree that an AI agent is when an AI system uses another system to accomplish a task. For instance, an AI agent could login into your favorite airline reservations system and buy a flight on your behalf.
DR "agents" run queries for you and do analyses of texts that they pull off the web. At the moment, no DR agents can log you into a paywalled system and purchase an article for you. THAT would be agentic, and (here's a prediction for you) it'll happen. But what we see today are very simple pulls from resources, not full agency.
Henk van Ess has done a nice piece of work contrasting 4 different "Deep Research" systems. To wit:
a. Gemini Advanced 1.5 Pro with Deep Research
b. OpenAI’s Deep Research (via ChatGPT Pro)
c. Perplexity’s Deep Research Mode
d. You.com’s Research Feature
As you might expect in these days of hurry-up and copy-what-everyone-is-doing, there will be more Deep Research systems in the next few weeks. (HuggingFace just announced that they made their DR system that "cloned OpenAI's Deep Research system" in just 24 hours. Reading their blurb on how they did this is interesting. (Even better: Here's their Github repo if you want to play with it yourself.)
I agree with much of Henk's analysis, but here's my take on the current crop of DR systems.
There are 3 big issues with DR systems from my perspective.
1. Quite often the text of the analysis includes all kinds of tangential / off-topic stuff. If you just blast through the text of the analysis, you'll miss that it's full of things that are true, but not really relevant. The writing looks good, but is fundamentally hollow because it doesn't understand what it's writing about.
2. If you ask a silly research question, you'll get a silly answer. As Henk points out in his analysis, asking a DR system about the "banana peel stock market theory" will generate plausible sounding, syntactically correct, but ultimately meaningless garbage.
Interesting: if you do an ordinary Gemini search with his question:
[Is there a correlation between banana peel speckle distribution and stock market fluctuations in countries that don’t grow bananas?]
you'll get a succinct "There is no known correlation between banana peel speckle distribution and stock market fluctuations in countries that don't grow bananas."
But if you ask Gemini 1.5 Pro with Deep Research you'll get a 2200 word vaguely humorous essay with data tables, a section on fungal diseases, and a list of stock market fluctuations. Even though ultimately the summary of the long-form essay is "Well, based on the available research, the answer is a resounding "probably not."" Gemini DR sure takes its sweet time getting there.
And Gemini handles this better than other DR systems, which generate profoundly idiotic (but polished!) answers.
Deep point here: If your question doesn't make sense, neither will the answer.
3. You have to be careful about handing your cognitive work off to an AI bot. It's simple to ask a question, get an answer, and learn absolutely nothing in the process.
This is a caution we've been saying in SRS since the very beginning. But now it's even more important. Just because something LOOKS nice doesn't mean that it's correct. And just because you've used the tool to answer a question doesn't mean you actually grok it.
What's more, if you haven't done the research yourself, you won't understand all of the context of the findings--you'll have none of the subtleties and associations that come with doing the work yourself.
You do NOT want to be in the position of this guy who was able--with the help of AI--to write a great email, but obviously has no idea how to actually do the thing he proposes to do. (Link to the Apple Intelligence video.)
Be careful with the tools you use, amigos. They can have a profound effect on you, even if you didn't intend for that to happen.
And keep searching--there's so much left to learn.
