Answer: How good is AI at recognizing images? What should you know?

Search by image is powerful... 

Remarkable desserts. What are they? 

.... but you need to know what it can do (reliably) and what it can't do (unreliably).  

Let's talk about what AI powered image search is capable of doing.  Here are the questions from last week:    

1. The image above (the dessert display) is from a cafe.  Can you figure out what KIND of desserts these are?  Yes, I know you can read the labels, but these are from a particular region of the world.  What kind of cafe is it?  (Image link to full image.)

The obvious thing is to do a Search-By-Image (which we last discussed in January, when searching for the  El Jebel Shrine, aka the Sherman Event Center in Denver.  That was just 11 months ago, but the world has shifted since then.  

We can download the image (with the link above) and do an image search (no longer called "reverse image search" since the function no longer does "reverse" image search, but tries to do an analysis of the image).  You'll get this: 

This is nice, but it's NOT a "reverse image search" in the way we used to think of it.  

To get that function, I'd use Bing image search, which gives you a result like this: 

In this case, there's no exact match for the image, but there are a lot of similar Middle Eastern restaurants and cafes full of yummy pastries.

On the other hand, the Google answer is interesting.  There's a good description of the contents of the pastry case, but over on the right side in the right-hand side panel you'll see a suggested "possibly relevant" link to Sana'a Cafe in Oakland.  

It's a bit of a spoiler, but this IS an image of the pastry case at Sana'a Cafe in Oakland, California!  The big question for us: How does it know?  This is definitely NOT the closest Middle Eastern cafe to my house (which is where I'm writing from).  

I checked to see if it was using the GPS location stored in the photo. 

(Remember that you can pull the lat/long of the image?  Previous SRS discussion about EXIF and the metadata attached to your images.)  

To check, I edited the image metadata to alter the lat/long and re-ran the query--and got the same answer!  

So what IS going on?  Answer: this image has a close-match to an image found in Reddit about the Sana'a Cafe in Oakland!  

Notice that you can get to the "similar images" section  by simply scrolling down the page to get to "Visual matches," where 3 of the top 5 visually similar images are from Sana'a Cafe.    (Note that these images are really similar to the way Image Search used to work--it would show you the nearest matches.)  

That's great--at least we now know how to get the old search behavior to function.  

Back on the first AI-augmented search page, you probably noticed that there's an option to "Show more."  Clicking on this button will give you a more detailed analysis of the image.  It looks like this: 

So.. yeah. Not a lot of help here--this is just a repeat of what we saw in the first frame.  But what happens if you click the "Dive deeper in AI Mode" button? 

Ooops. Now Image Search is going off the rails.  How does Google know that it's the Levant dessert cafe and bakery?  Completely unclear.  And no amount of asking it would give me any useful chain of reasoning.  

Rather than using plain Google Image Search, I thought I'd give Gemini a chance.  One MIGHT hope that the answers would be the same (it's the same company, right?). So I uploaded the image to Gemini and asked it to describe the image.  No surprise, it gave me more-or-less the same answer.  

But when I asked Gemini a follow-up questions [where is this dessert case located] the Google train goes off the rails and into the river where it crashes and burns.  

This is the equally incorrect response, although incorrect with a florid explanation that's completely wrong: 

As much as I admire the idea of reading the reflected text of the logo (which reminds me of what we did in SRS 2012 ("Where are you?"), in this case, it's totally wrong!  I can't see the "Kunafas" anywhere in the image (can you?).  

So I asked Gemini where the "Kunafas" came from.  Here's what I got when I asked: 

Seems good, right?  But let's look at the highlighted region carefully, shall we?  Here, I put the original image and the Gemini-created image side-by-side.  

As you can see, the "reflected letters" are clearly--at least to you and me--the letters of the cafe's name, Sana'a.  The "F A N U K" are all hallucinated.  

Even more bizarrely, I was curious and re-did the original query on regular Google Image Search, using the same image as before and asked Google Image search to describe the image.  This time, it suggested that the place might be the Sana'a Cafe... but again, not reasoning about why.  I assume it's using the "related images" feature and extracting the name from the Reddit thread images.  This is bizarre because it's NOT the same answer from earlier!  

Bottom line: You absolutely have to check everything that Image Search tells you.  Don't just accept it as truth--it could be very far from the truth.  

2. Here's a photo I took while on a walk in San Francisco the other day.  What a strange, strange place!  It's clearly supposed to have a statue on top of the pedestal.  What happened here?  Why is it bereft?  (Image link)  

I did the same process as before:  regular Image Search on Google and get this as an answer: 

The AI overview is completely wrong.  This is NOT at Lands End park at all... everything in this result is wrong.  

On the other hand, the "Visual matches" section actually gives good results.  This IS "Mount Olympus" (the San Francisco version).  

So, let's try again with the fancy Gemini-powered AI image identification process.  What do we get here? 

The first answer ("...likely the Stairs to Mount Olympus Park in San Francisco..") IS correct, while the "another possibility is the One Thousand Steps Beach Access in Santa Barbara" is quite wrong.  

As before, if you ask Gemini directly (by uploading the picture and asking "where is this image"), you get another kind of wrong answer: 

At least it got the trees right (they are Monterey Cypress), but everything else is seriously wrong.  

First off, there IS NO Hilltop Monument at The Sea Ranch.  (I've been there quite a bit, and I'm 99.9% sure such a place doesn't exist.)  Google might mean the Sea Ranch Chapel, but it's not called the Hilltop Monument, and it's not on a hilltop in any case--it's in the flatlands.  

I thought maybe I'd give ChatGPT a chance, but that didn't work either: 

Again with the Lands End?  The only connection is the Lands End also has a lot of Monterey Cypress, but there's no other connection here.  And there IS a monument to the USS San Francisco at Lands End, but again, it has nothing to do with this picture.  Hallucinations abound.  

And, once again, the "Visual Matches" section of the SERP gives you a much better result than the AI parts of the result: 

But you, dear Human, can easily pull the GPS lat/long from the EXIF metadata to find this in Google Maps: 

And then, a regular Google search [ Mount Olympus Park San Francisco ] will teach you that Mount Olympus was a park in more-or-less the center of San Francisco, with a pedestal, atop which stood a dramatic statue, "The Triumph of Light." Mysteriously, the statue (made of bronze and weighing probably 500 pounds) vanished from the pedestal years later and has never been found.  (See the backstory here at FoundSF.org) 

The statue that was there: 

Mount Olympus in SF, with the original statue that mysteriously disappeared sometime after 1955. P/C San Francisco History Center, San Francisco Public Library, via OpenSFHistory.org 

And nobody knows where--or even exactly when--the statue disappeared.  The city took it's collective eye off the ball and it just kind of went-away one day in the mid-1950s.  

Bottom line: Don't trust the AI analysis.  Do the research yourself. 

3. Here's a great picture of a cloud that Regular Reader Ramon sent in for identification.  What's going on here?  (Image link)  

P/C SRS Regular Reader Ramon

A regular Google Image search tells us that this is a fallstreak hole, also known as a "hole punch cloud."  

As you'd expect, I checked this out by doing other searches (e.g., for [fallstreak cloud]) and looking at the collection of remarkable and beautiful photos.  In this search, the AI result and "Visual matches" images are all pretty good.  

And now we know that the fallstreak cloud is caused by supercooled water in the clouds suddenly evaporating or freezing, possibly triggered by passing aircraft passing through the cloud and causing a chain reaction. Such clouds aren't unique to any one geographic area and have been seen in many places.  

Bottom line:  This worked quite well--not a huge surprise as the image is very visually distinct and there are literally thousands of posts with images describing what this is.  

4. This little bridge is in a lovely town somewhere in the world.  Can you figure out where it is, and when it was built?  (Image link)

This is a case when image search works quite well.  Luckily, this is a famous bridge with LOTS of photos taken over the years.  

Yes, it's the Pinard Bridge, located in Semur-en-Auxois. It (and much of the town) date to the 12th century.  But it's really hard to determine when it was first built.  It will probably take some time searching in old French histories to figure out the original date. But since it's in the river valley that historically floods, it's been rebuilt many times.  

Regular Reader Arthur Weiss points out that the city's website of Semur-en-Auxois tells us that  "The Pinard bridge, or Pignard on the Belleforest view, provided access to the Pertuisot mountain pasture. It was destroyed or extensively damaged on several occasions by floods, including those of 1613, 1720, 1765 and 1856."

(I also found this website with the search [ville-semur-en-auxois pont pinard] -- this is one of those cases when searching in the local language really helps.)  

So while the date of first construction was probably in the 12th or 13th century, it's been rebuilt so many times that little of the original bridge is now left in place.  It is, as we would say today, and example of the Ship of Theseus (if Theseus' ship is replaced plank by plank over a long time until all pieces of wood have be replaced by newer wood, is it the same ship?).  

Search Research Lessons

1. Be very, very cautious about AI generated results.  As we saw, the results can be very, very wrong. My advice: Try the AI methods, but double-check everything.  You cannot trust that the answer is correct.  

2. Note that "Visual Matches" section of Image search (often below the fold) has the "old style" most similar images from the web.  That section also often has great clues to the actual thing you seek.  Be sure to check that part of the search results as well.  

Keep searching! 

SearchResearch (11/13/25): How good is AI at recognizing images? What should you know?

 Recognizing images is an impressive AI feat.  But... 

Remarkable desserts. What are they? 

.... it's true that the state of the art of image recognition has changed over the past several years.  It gets better, it gets worse, the functionality changes, some things are removed, others are added.  

But it's still an amazing thing... IF you know what works and what doesn't now.  I'm afraid that means you have to stay up on what's going on in the world of image search.  So let's dive into it... 

Here are a few images that I'd like for you to identify--the key question for each is what's going on in this image?  What is it?  (And if you can, where is it?)  

For each image I've given you a link to the FULL image (no sneaky reduction in resolution or removal of metadata, as our blogging tools tend to do).  I recommend you use that image for your search.  

1. The image above (the dessert display) is from a cafe.  Can you figure out what KIND of desserts these are?  Yes, I know you can read the labels, but these are from a particular region of the world.  What kind of cafe is it?  (Image link to full image.)

2. Here's a photo I took while on a walk in San Francisco the other day.  What a strange, strange place!  It's clearly supposed to have a statue on top of the pedestal.  What happened here?  Why is it bereft?  (Image link)  

3. Here's a great picture of a cloud that Regular Reader Ramon sent in for identification.  What's going on here?  (Image link)  

P/C SRS Regular Reader Ramon

4. This little bridge is in a lovely town somewhere in the world.  Can you figure out where it is, and when it was built?  (Image link)

The point of this week's Challenge is to give you a bit of familiarity with the different image reco tools.  They're sometimes called "Reverse image search" tools, but as you'll find out, they have very, very different capabilities.  

When you write in to let us know what you found, be sure to (a) tell us what tools you tried, (b) if they worked well, and (c) whether or not you find the answer believable. 

Next week I'll write up my findings and summarize what everyone else found... along with a description of the tradeoffs involved in the different tools.  

Keep searching! 

SearchResearch (11/5/25): Pro tips on using AI for deep research

A few friends... 

Gemini's conception of [hyperrealistic image of scholar doing deep research].
Not sure it's hyperrealistic, but definitely interesting. 

... have recently written posts of their own about using AI for deep research. Since they've got some great nuggets, I'm going to leverage their writings and give a quick summary of the top methods for doing high quality deep research with LLMs. 

In this post, I'm drawing extensively on a post written by Maryam Maleki (UX Researcher at Microsoft) for people doing product research:  How to Do High-Quality AI Deep Research for Product Development  Here, I've generalized it a bit and given it my own flavor.

Here are the top few tips about getting Deep Research mode to work well for you:  

Be clear about what you want.  

Keep in mind: You want credible content. Prompt it that way. 

In order for the AI to work, you need to tell it what kind of sources you think are reliable and credible.  If you can, give it a list of several resources as guidance. 

In these patterns below, items in { } and italics are variables.  You need to pop in the values you need to get the effect you want.  

Pattern:   

[ Do deep research on {TOPIC}. Generate {n} credible sources with links that can be used for this research.

Prioritize: {BOOKS / ACADEMIC PAPERS / CASE STUDIES}

For each source, provide: the Title, the URL, a short snippet about why it's relevant, tell me the Source type. ] 

Example: 

[ Do deep research on Rocky Mountain locusts. Generate 10 credible sources with links that can be used for this research.

Prioritize: academic papers

For each source, provide: the Title, the URL, a short snippet about why it's relevant ] 

Doing this in Gemini will create a 4,000 word essay about Rocky Mountain Locusts.  It will ALSO give you section VII, which has Ten Credible Sources for Rocky Mountain Locust Research.  It also creates a reference list for the entire document, with section VII containing the best of the entire list.  

By contrast, doing this in ChatGPT 5/Thinking or Claude Sonnet 4.5 gives you exactly what you asked for--they give you the list-of-ten.   

Review the AI-generated results for quality 

I note in passing that the Gemini-created document is pretty good, but the list of 10 papers was a little mixed in quality.  (One paper was very tangential, one paper was just a link to Wikipedia, and one paper wasn't accessible at all.)  I clicked through all of the links to verify that they were real and on-target.  

If the results aren't what you want, feel free to iterate until you get the result quality you need.  

Press enter or click to view image in full

Ask for contrary points of view
(don't just confirm!)  

Research isn’t just about collecting references — it’s also about understanding the space, both in terms of what you know and what counterarguments you might want to consider. 

In reading through the Rocky Mountain Locust collection, you'll notice that one of the main hypotheses about the disappearance of the locust is that the rangeland where it lived and bred was increasingly plowed up for farmland.  

You should ask about other opinions:  

Pattern:   

[ GIve me different explanations for {TOPIC}.  Are there other points of view that have been considered in the literature?  

For each source, provide: the Title, the URL, a short snippet about why it's relevant. ] 

Example:

[ Give me different explanations for why the Rocky Mountain Locusts disappeared.  Are there other points of view that have been considered in the literature?  For each source, provide: the Title, the URL, a short snippet about why it's relevant. ] 

Interestingly, Gemini merely did an okay job of this step--ChatGPT was reasonably good, but Claude did a spectacular job of highlighting 11 different hypotheses about what happened.  (To see Claude's output, here's the document.)  This also suggests that you should get multiple AI opinions to improve the quality of your research!   

Double Check Everything

We still live in a hallucinatory world. As great as AI generated content is, I still double check everything.  In her post, Maryam has a great set of questions (below).  This is what is on my mind as I read through EVERY claim and EVERY linked document.  You should too.  

• Source Quality — Is it recent, reputable, and methodologically sound?
• Fact Containment — Only use approved notes/sources. 
• Triangulation — Every claim needs at least two independent sources.
• Original-Source Tracing — Don’t rely on LinkedIn slides, Twitter posts, or a quote in a blog. Find the earliest credible publication.
• Hallucination Sweep — Audit the final draft. Remove or qualify any claim not directly supported.

Search Research Summary

When using AI for deep research, keep in mind 3 heuristics: 

1. Be clear about what you want.  Not just in content, but in form and quality.  Be explicit--give examples--ask for everything you want. 

2. Review the results for quality. Do this step immediately, and change the prompt if need be to get what you really seek.  Iterate!  

3. Ask for contrary points of view.  Don't give in to confirmation bias--proactively ask about other perspectives on the questions you're researching. 

4. Double check everything.  No surprise here, but be sure to leave enough time to do this.  Don't just copy/paste what you've found. 

 

Thanks again to Maryam for her excellent post.  

Keep searching!

SearchResearch (10/29/25): The 1 trick you need to know to use AI for deeper, better reading

 I absolutely adore... 

P.G. Wodehouse.  P/C Wikimedia

... the writings of P. G. Wodehouse.  Whenever I need a lift in the old spirits, I pluck a volume from the bookshelf of Wooster and Jeeves, I read a bit, and in the blink of an eye, all is right with the world.  As Wodehouse might say, God is in His heaven and the celestial choirs sing again.  

If you don't know Wodehouse, drop what you're doing and read a short story or two. Better yet, pick up a Wodehouse novel and dive in.  

I'd recommend Right Ho, Jeeves, which is an excellent place to start.  

The writing is droll and the language--especially the language--just tickle my humorous bones.  

BUT, Wodehouse is satirizing the language and behaviors of the early 1900s upper class.  They are a rich vein to mine, but roughly once each page, there is a phrase or word that escapes my understanding or offers up a nuance that completely misses my brain.  

For instance: 

Butter-and-egg man (An investor with a lot of money)

Absquatulate (To depart suddenly or abscond)

Cattywampus (Used to mean something that was directly across from something else, as opposed to its modern meaning of being askew or in disarray)

Those are fairly easy to look up.  But the more tricky phrases are things like: 

"Only that she’s a blister.”

Or... 

"Deprived of Anatole’s services, all he was likely to give the wife of his b. was a dirty look."  

I know what a blister is, but the obvious definition makes no sense here.  And what is "...the wife of his b."?  That's clearly not the end of a sentence, but feels like an abbreviation for something--but what? 

Here's where your friendly, local LLM comes in handy.  Here's what I did to figure out each expression:  I asked an LLM (Gemini in this case) to explain it to me in the context of the book... 

And when you need to be even more specific, give the name of the story in the context you provide to the LLM.  

In both of these cases, it's not clear that any amount of contextual reading would have taught me these meanings.  

This is a brilliant use of an AI to augment your ability to deeply read a text.  

On the other hand, use caution:  AI still makes mistakes, and they can be subtle. 

Here I asked a question about the mention of a device in a book written about the same time as Wodehouse: 

This completely checks out.  (Of course I double check everything.  Don't you?) The Veeder box is indeed a type of odometer made at the time.  

However... see this next part of the explanation: 

That mention of "By the time Evelyn Gibb and her husband were bicycling the West Coast in 1909..." is completely made up.  The book is NOT about Evelyn Gibb and her husband, but is about Vic McDaniel and Ray Francisco, friends who cycled 1,000 miles from Santa Rosa, California, to Seattle, Washington, for the Alaska-Yukon-Pacific Exposition. The author (Evelyn Gibb) is Vic's daughter, not his wife.  

SearchResearch Lessons 

1. Using an AI to give insights into obscure texts can be incredibly handy.  By virtue of having ingested so much text, an AI can often give you a perspective about a fragment of text that you don't understand. 

2. CAUTION:  Check everything--there are still hallucinations about!  Double check everything!

Hope you find this useful SRS method!  

Keep searching.