SearchResearch (7/3/26): What you need to know about image search (3/3)

 The most important thing to know about image search... 

P/C Gemini. Prompt:
[usually it works, but sometimes it gets it wrong]

... is that Image Search works pretty well, but is not perfect.  One of the skills you have to develop, as a user of online search tools, is to understand their limits.  When does the AI work well, and when does it fail?  

It's pretty easy to find places where Search-by-image doesn't work.  Many of them involve identifying plants.  

I've written about this before here at SearchResearch. What are those plants? What I said then is still true.  Yes, the AI is improving, but it's still not quite perfect.  

You really need to know this, as it's probably going to be this way for a while.  

Here's today's example.  While walking at the Googleplex, I saw a tree with browning leaves.  Here's the photo I took: 

But when I asked Google Lens what this tree was, here's the answer I got... 

Thing is, I know this particular tree REALLY well--I walk past it nearly every day, spring, summer, fall, and winter. I know for certain that this a California Buckeye (Aesculus californica)--you can get all of the details from the authoritative source, CalFlora.  

But Google search-by-image (aka Google Lens) gets it wrong. I can't really blame it because it looks a lot like a California Sycamore (Plantanus racemosa) in its fall foliage.  But it's only early July, so the California Sycamores look like this: 

A California Sycamore at the beginning of June

What Google Lens doesn't know is that California Buckeyes typically brown up and drop their leaves in mid-summer. A quick search for [when do California Buckeye leaves turn brown] gets you to the University of California Master Gardener's page where you can confirm this.  

Google Lens identification got it wrong. 

Another non-botanical example is this beautifully carved slate plaque that I found as part of an art installation.  

I love the texture and the script, but... I can't quite read it!  Have I had a stroke, or is this in some other language? 

Upload the image to Google looking for an analysis.  What IS that text? Here's what Lens tells me... 

This is interesting, but not particularly helpful. 

A trick worth knowing:  You can scroll up a bit in the search window and--lo & behold--a new query box appears!  It will let you ask an additional question of the search: 

That's great!  I asked the obvious question:  [what language is this text?]   Google replied that it was Russian!  

That's a surprise--it doesn't look like Russian to me, but then I don't know what cursive Russian looks like.  

But I do have a Russian friend who reads and writes with fluency. When I asked, she was astounded that I would ask such a silly question.  "Of course this isn't Russian!"  But she DID know about the plaque, telling me that it was part of an art installation and that she knew who the artist was!  (It sometimes pays to ask humans about tricky questions...)    

Turns out that the artist is Sarah Stiles who makes "Cursive Binary" with her handwriting modified by AI.  

Since this doesn't look like Russian, where did Gemini get the idea that this was Russian?  

Well, that's interesting.  If you look at Stiles website, you'll see that she's of Kalmyk heritage.  (That is, from the Republic of Kalmykia, a federal subject of southwestern Russia situated between the Caspian Sea and the lower Volga.)  

I can't prove it, but I suspect there's a bit of AI leakage from the original query image to the question-answering component.  

SearchResearch Lessons 

1. Search still makes mistakes; AI search does too.  Be sure to check your work. Always.  

2. If you ask the AI to go further, it will.. but that's when you're asking for hallucinations to happen. 

3. You need to know what image identification is good at, and what's not good at.  Plants are a difficult case because they often require very close attention to detail in order to tell one apart from another.  The ID might be close, but not quite right. (And as you know, never use AI to identify mushrooms.  This is another case where the fine details really matter.)  

4. Most search-by-image systems don't let you identify people by name.  Of course, your personal content is different. Both Google Photos and Facebook let you identify people in your collection of pix.  

5. Image recognition is really good at commercial products.  Makes sense, since most of the training material is commercial in nature.  Even partial or incomplete images can often be recognized.  

Keep searching!