A student asked me a simple question...
... and I couldn't come up with an answer that was compelling to me (although I think the student was okay with my answer).
The question was: "You said that you use regular Google for some kinds of research questions, and LLMs for other types of research questions. How do you know when to use which?"
I gave the student an answer (because that's what professors do), but I had a little vague feeling in the back of my mind that this wasn't a very good answer.
So I thought I'd ask the collective wisdom and insights of the SearchResearch team. Here's the Challenge for the week:
1. How do you know when an LLM AI system will give a good answer to your question? How would you characterize a research question that's really good for AI versus a research question that you'd just use a "regular" search engine for?
I think what I'm looking for is a clear description of when an AI is most likely to give an accurate, high quality answer? By contrast, I think I know how to say when I'd use a search engine, but it's harder to describe the kinds of questions that I think an AI would do poorly.
Can you help me think through this Challenge? What kinds of research questions do YOU ask your AI... and have confidence that you'll get a decent answer? (And conversely, what kinds of questions do you NOT ask your favorite AI?)
Remember that a couple of weeks ago I posted about how terrible the various AIs are at generating diagrams? Well, there's one answer about a kind of question to not ask an AI: Don't ask them to create a diagram for you.
Here's Gemini's attempt at creating a diagram of a toaster.
Yeah. I have no idea what any of those parts are aside from the crumb tray. What's a Contreue or a Frerriod?? Maybe this is the way toasters look in a far distant galaxy, but not in any country (or language) on Earth! This toaster would be a disaster in reality.
So there's one part of the answer: asking an AI to create a diagram for you is a truly terrible idea. (And under no circumstances should you ask for a diagram of something you don't really understand.)
Let us know what you discover--post your observations in the comments, and I'll summarize them (and my thoughts) about this next week.
Keep searching.
In my case I normally use regular search engine. And now we have with Google the AI created overview.
ReplyDeleteWith that I solve most of my searches.
When I use LLMs? Mostly when needed a "summary." As an example, the games of soccer of specific team.
Also when I don't have a clue about how to start searching on my own. Or when I need or want to do a follow-up question
I think LLMs are also helpful with images. I tried to find something and still nothing. But I think it's something that can't be done so it's not "searchable.'
I feel that for some reason the easy things that we already had with regular search doesn't work with AI. The other day I was listening a song. Asked Gemini what song it was. The answer was: Not media playing. So I went to regular Google. Clicked the microphone. Listening and my song was found. I know that I probably did something wrong and Gemini can do that too. But it's complicated.
I'm looking forward to read the comments and answer from advanced users, complicated searches and questions and of course from Dr. Russell.