A student asked me a simple question...
... and I couldn't come up with an answer that was compelling to me (although I think the student was okay with my answer).
The question was: "You said that you use regular Google for some kinds of research questions, and LLMs for other types of research questions. How do you know when to use which?"
I gave the student an answer (because that's what professors do), but I had a little vague feeling in the back of my mind that this wasn't a very good answer.
So I thought I'd ask the collective wisdom and insights of the SearchResearch team. Here's the Challenge for the week:
1. How do you know when an LLM AI system will give a good answer to your question? How would you characterize a research question that's really good for AI versus a research question that you'd just use a "regular" search engine for?
I think what I'm looking for is a clear description of when an AI is most likely to give an accurate, high quality answer? By contrast, I think I know how to say when I'd use a search engine, but it's harder to describe the kinds of questions that I think an AI would do poorly.
Can you help me think through this Challenge? What kinds of research questions do YOU ask your AI... and have confidence that you'll get a decent answer? (And conversely, what kinds of questions do you NOT ask your favorite AI?)
Remember that a couple of weeks ago I posted about how terrible the various AIs are at generating diagrams? Well, there's one answer about a kind of question to not ask an AI: Don't ask them to create a diagram for you.
Here's Gemini's attempt at creating a diagram of a toaster.
Yeah. I have no idea what any of those parts are aside from the crumb tray. What's a Contreue or a Frerriod?? Maybe this is the way toasters look in a far distant galaxy, but not in any country (or language) on Earth! This toaster would be a disaster in reality.
So there's one part of the answer: asking an AI to create a diagram for you is a truly terrible idea. (And under no circumstances should you ask for a diagram of something you don't really understand.)
Let us know what you discover--post your observations in the comments, and I'll summarize them (and my thoughts) about this next week.
Keep searching.
In my case I normally use regular search engine. And now we have with Google the AI created overview.
ReplyDeleteWith that I solve most of my searches.
When I use LLMs? Mostly when needed a "summary." As an example, the games of soccer of specific team.
Also when I don't have a clue about how to start searching on my own. Or when I need or want to do a follow-up question
I think LLMs are also helpful with images. I tried to find something and still nothing. But I think it's something that can't be done so it's not "searchable.'
I feel that for some reason the easy things that we already had with regular search doesn't work with AI. The other day I was listening a song. Asked Gemini what song it was. The answer was: Not media playing. So I went to regular Google. Clicked the microphone. Listening and my song was found. I know that I probably did something wrong and Gemini can do that too. But it's complicated.
I'm looking forward to read the comments and answer from advanced users, complicated searches and questions and of course from Dr. Russell.
I'm thinking about Grok and the controversy created these past days. How this "behavior" affect the rest of the LLMs? In this case the owner point of view is there, allegedly. But I think people will start not liking this technology after that
Deletehi Ramon - if you haven't seen... flux inbound
Deletehttps://x.com/xai/status/1943158495588815072
https://youtu.be/LTrd6sOqIhg?si=hsUJimxFFfddBG1k
Thanks Remmij!
DeleteIt's scary and hopefully, if it's true what Glenn Beck video said, they use it for the good of everything.
I didn't know about AGI.
For me, it sounds like fantasy. However, knowing Mr Musk, probably Grok 4 is even more terrifying than what he said.
Dr. Russell, Remmij a question. What makes Grok different than Gemini, Perplexity & Chatgpt ?
I have to say I haven't used it yet. And SRS Challenges tests have shown it is, now, as good as others
Elon's expressions are cryptic... he does say possibly
Delete"terrifying", but yeah, well so...
and there is that $300/mo. subscription fee...
a small price to pay to know it has all zoomed past..
the/a future is not cheap. but omniscience never is..
that is kinda an awkward pair of couches.
https://x.com/xai/status/1943158495588815072
https://www.teslarati.com/xai-launches-grok-4-supergrok-heavy-subscription-details/
things will be changing rapidly.
a generic toaster text description and graphics - some of the diagrams are better than others... progress?
Delete🤔🍞
(Gemini 2.5 Flash)
diagrams:
https://i.imgur.com/prtl5fI.jpeg
description:
"Generic Toaster Parts Diagram
A typical toaster, designed for browning slices of bread, consists of several key components working in unison:
Toasting Slots: The openings at the top where bread slices are inserted. Most common toasters have two or four slots.
Heating Elements: These are typically nichrome wires that glow red hot when electricity passes through them, radiating heat to toast the bread. They are usually arranged in a grid pattern around the slots.
Lever/Plunger: This mechanism is used to lower the bread into the toasting slots and engage the heating cycle. It often has a spring-loaded design.
Crumb Tray: A removable tray located at the bottom of the toaster designed to catch crumbs, making cleaning easier.
Control Dial/Slider (Browning Control): Allows the user to select the desired level of toasting (light to dark). This typically controls the duration of the heating cycle.
Cancel Button: A button that immediately stops the toasting cycle and ejects the bread.
High-Lift Lever (optional): Some toasters have an extra lift feature on the lever to raise smaller items (like English muffins) higher for easier retrieval.
Bagel/Defrost Buttons (optional): These buttons adjust the toasting cycle for specific items.
Bagel: Often toasts only one side of the bagel.
Defrost: Adds extra time to toast frozen bread.
Power Cord: Connects the toaster to an electrical outlet.
Housing/Casing: The outer shell of the toaster, typically made of plastic or metal, which encloses and protects the internal components.
Internal Wiring and Circuitry: The electrical components that connect the heating elements, controls, and power supply, managing the flow of electricity and the toasting cycle.
Thermostat/Timer: An internal component that works with the browning control to regulate the toasting time and temperature, ensuring consistent results.
Eject Mechanism: The internal system that lifts the bread out of the slots when the toasting cycle is complete or cancelled."
Ramon - ran across these - you may be interested:
Deletehttps://ai-2027.com/
Three-Tiered AI Framework:
https://shorturl.at/XSkvn
https://www.tandfonline.com/doi/epdf/10.1080/09540261.2025.2507284?needAccess=true
...getting back to Dan's question...
have a good weekend ;^]
(an example of ANI - artificial non-intelligence)
not sure what their range is - may not be in Mexico...
https://i.imgur.com/XtKDmyj.jpeg
fwiw - from Gemini Overview
https://smythos.com/developers/ai-models/grok-origin/#:~:text=Heinlein%20crafted%20'grok'%20as%20part,with%20the%20subject%20at%20hand.
"The word "Grok", and the namesake of the AI chatbot developed by xAI, was coined by science fiction writer Robert A. Heinlein in his 1961 novel Stranger in a Strange Land. In the novel, "grok" is a Martian term that signifies a profound and intuitive understanding, going beyond mere intellectual grasp to involve an emotional and empathetic connection with something.
Heinlein's concept is nuanced; it doesn't just mean "to understand intuitively or by empathy", but also to merge with, or become part of, the thing understood. The book explores the depths of this understanding, even associating it with the idea that to truly hate something, one must first deeply understand and, in a sense, become one with it.
The word has since been incorporated into English dictionaries and has found usage in various contexts, particularly within the computing and hacker subcultures, where "to grok" a piece of code or technology signifies a deep and thorough understanding of its inner workings and implications. "
Hi Remmij :)
DeleteThank you! I'll visit the links you shared. I wish you a happy weekend too.
300 is a lot! However, it's cheap if like in the video is said you are a company or investor and AI gives you a way to create new element, new medicine or something that makes them with more millions. I have to say that I don't think that will happen. Why? Because if any AI is capable to obtain those "miracles" I am sure the owners or the government will get them before regular people can know about it.
I hope that technology helps everyone and the world. As Dr, Russell say: If that happens we will read about it in all places
I've also thought about this. Not sure where this came from (and I adapted it anyway) but the easiest way (for me) is to categorise questions into one of three types:
ReplyDelete1) Navigational queries e.g. how to get to a company website, find a contact address, etc.
This is an easy search engine win. It's a simple answer that a website link will solve.
2) Simple informational queries: what's tomorrow's weather in Los Angeles (or Zurich). Who is the current CEO of (or President of)...
Generally search engines win - although sometimes AI may be better if it's slightly more complex or esoteric. This search engine win gives a reasonable answer for toasters, .
3) Exploratory queries where there is no single or simple answer and the research may involve
a multi-step processes to answer. For such questions, AI wins (backup by checks using conventional approaches).