Answer: To search, or to AI? That is the question...

The question is--what works? 

If you've been using online search tools for any amount of time you know that which tool you use, and how you use it can make a huge difference in the quality of your results.  

This week I Challenged you to find what each of these images is really about.   When you try to find out the story of these images, where you able to use "regular" Search-by-Image, or did you use AI image descriptions?  If so, what prompt did you use, and how well did it work for you? 

1. While traveling in northwest Switzerland, I saw this emblem all over the city I was visiting.  Can you tell me what city I was in?  Just as importantly, what IS this thing? 

For each of these images I used 2 different methods: (1) regular Google Search-by-Image, SBI (you know, upload the image and let it search--aka Google Lens), and (2) upload the image to an LLM (either Gemini or ChatGPT) and ask it to identify what it sees in the image. 

Image B1: On a box of chocolates... 

Image B2: Carved into a stone wall

Image B3: Seen on a parade float

Here's what I found: 

B1 (gold symbol on black/white background): This image is very focused on the object of interest and SBI does a much better job.  Here's what the Google Lens (mobile) interface looked like when I asked about this: 

Mobile Lens search for B1 image

It STILL doesn't tell me what it is, but at least we learn yet another potentially interesting word: Baslerstab.  Searching for this leads to: 

We see that "Baslerstab" is the name of the local Swiss-German newspaper AND that "baselstab" is the name of the symbol!  (German note:  "Basler" means a person or thing from the city of Basel.)   Clicking through on the Wikipedia link takes us to the Baselstab article, telling us a bit more: 

"The Baselstab or Baslerstab is the name given to the shepherd's staff or bishop's staff in the coats of arms of villages and towns in the Basel region in Switzerland. The crosier of the Prince-Bishop of Basel can be found in the coats of arms of the city of Basel and the cantons of Basel-Stadt, Basel-Landschaft, and Jura. The crosier of Basel-Stadt is black, while those of Basel-Landschaft and Jura are red."  

Excellent.  Now we know that this symbol is the emblem for Basel and is an abstract version of a bishop's crozier.  

Since I told you that the Baslerstab was on a box of chocolates, I was wondering what the connection was between this symbol and chocolate.  Reasonable question for Gemini, yes? 

Unfortunately, Gemini seems to fall into a hallucinatory rabbit hole with this answer. Despite what Gemini says in its answer, as far as I can tell, there is NO connection between Lindt (a large chocolate company in Zürich) and the actual chocolate company that uses the Baslerstab symbol (which is called Laeckerli-Huus, headquartered in Basel).   Aaarrghhh!  Everything in this answer below about the crozier and Lindt is wrong.  

Since that didn't work well, I did the obvious query on regular Google: 

If you click on that first link you'll see exactly the image of the entire box of chocolate and cookies, a nice example of the baslerstab being used as a design element.  

 

That's MUCH better than Gemini.  Another example of doing a simple Google search that works out much better than asking an AI.  

For the record, ChatGPT didn't know either... and also completely hallucinated a connection between the baslerstab symbol and coat of arms of the canton of Fribourg, Switzerland where the chocolate maker Maison Cailler is based.  Utter balderdash.  

B2 (symbol on wall): This was the symbol carved into a stone wall in the city's center.  Sadly, nothing I could do with SBI would work.  Google failed, Bing's SBI failed, and even focusing in tight on the symbol didn't help.  Search-by-Image just couldn't handle this.  

When I turned to Gemini with the wall-carving image, and asked [ identify this symbol ], Gemini replied: 

"The symbol shown in the image appears to be a heraldic shield with a stylized bird's head and neck facing left, and a lozenge (diamond shape) below it."  

Not quite.  

However, Gemini also said (quite reasonably):  

"To understand the full meaning of this symbol, you would need to know the context in which it was found.  Here are some questions to consider: Where was the image taken? The location could provide clues about the origin of the symbol and the family or institution associated with it."  

To which I answered, [ the image was taken in Basel, Switzerland ] to which Gemini replied: 

"Okay, knowing that the image was taken in Basel, Switzerland, provides some very important context. Here's how that helps us understand the symbol:

Basel's Heraldic Traditions:

Basel has a rich history of heraldry. Notably, the city is strongly associated with the "Baselstab" (Basel staff), a stylized bishop's crozier. This is a very prominent symbol in the heraldry of Basel-Stadt (the city of Basel) and Basel-Landschaft (the surrounding canton).   

Another significant heraldic figure associated with Basel is the Basilisk.

 This mythical creature is frequently seen in Basel's heraldic displays."  

Which tells me that this is a baselstab (the crozier symbol) AND that the City of Basel also uses a basilisk as a heraldic element.  

  

But of course I had to check this as well... luckily, it passes the sniff test.  

When I checked with ChatGPT, it also gave me a poor result: 

ChatGPT trying to identify the carved symbol.

But since Gemini needed to know where the image was from, I also told ChatGPT that this image is from Basel.  Guess what?  A new answer emerged!  

After I told ChatGPT the image is from Basel, it was able to figure out what it was.  

Lesson learned here--location matters!   

So I tried again with Google SBI.  Turns out it DOES matter!  

It makes complete sense--given a random image from somewhere in the world, adding LOCATION information would help any identification process (including humans!) to figure out what a symbol is.  

B3 (dragon with shield): Going through the process again... and once again, SBI didn't work, even when I cropped to just the symbol. Here's the "regular" SBI search:

 

Click to see full-size

It correctly identified the dragon-like creature as a basilisk, but that's not what I'm interested in--I want the symbol on the shield. 

Fortunately, Google Lens now has the ability zoom in on just the key feature by dragging the white rounded angle brackets--in this case, to focus in on the symbol.  When you do that, you'll get an okay search--the first two results are good, but the rest are terribly off-topic! 

The good news is that you learn that it's the city symbol of Basel, but you won't find out WHAT it is!  

If you ask Gemini to [ describe the image ], you'll get a nice description:  

"This image shows a close-up, angled shot of a bronze sculpture of a griffin holding a shield. The griffin, with its eagle-like head and wings, is the dominant feature, positioned in the left half of the frame...The shield the griffin is holding is prominently displayed in the center of the image. It features a unique emblem, possibly a heraldic symbol, with a curved, stylized design."

But no amount of me asking what that symbol is could convince Gemini to give me a real answer.  Not a great result.  Even when I gave it the cropped version of the image (showing just the symbol itself), Gemini couldn't say what it was.  

By contrast, ChatGPT did much better:  When I asked [ what's the symbol in the center of the shield ] ChatGPT responded with: 

"The symbol in the center of the shield is a heraldic emblem known as a "crosier," which is stylized like a pastoral staff with a curled or spiral top. It resembles the ceremonial staff traditionally carried by bishops or abbots, often symbolizing religious or spiritual authority. The depiction on this shield is stylized, emphasizing the curved top and decorative elements typical of such symbols."  

So now we have a term to look up: "crosier."  A quick search takes me to Wikipedia where I learn that the word is usually spelled "crozier" and it's a stylized shepherd's crook, which is ALSO the symbol for the city of Basel, as you can see in the image below:

P/C Wikipedia

But then I took my own advice and added Basel Switzerland to the SBI search, and as expected, it really improved things: 

SO... I took my own advice AGAIN... and re-ran the very first query of Gemini ADDING in the information about [...this image was taken in Basel Switzerland].  Voila!  Here's the answer:  

Which is a pretty good result.  

BUT... you have to know to prompt it with all of contextual information!  

I'll tell you the ground truth:  I took all of these images while visiting Basel for Fasnacht--huge parades with floats (like the one above), and repeated crozier symbols everywhere.  Since I was literally seeing this just about ever time I looked up, I was curious what it was.  B3 was taken during a Fasnacht parade as a float slowly drifted by me, the box of chocolates was something I bought, and the stone carving was taken as I walked around the city at 4AM for the Morgestraich.  

2. In open green spaces all over Zürich I keep seeing this flower popping up--they're growing everywhere in vast numbers.  What is it?  Again, how do you know?  What search method did you use? 

Image P1: A few flowers carpeting the lawn. What are they?

I took this photo at a random lawn in Zürich because I was curious--any open green space seems to be blanketed with these flowers!  What are they?  

Since I was walking around, I did the obvious Google Lens search: 

 

As you can see, it calls it a gentian.  Well... notice how Lens has selected an individual flower in the middle?  That does kind of look like a gentian, but I know gentians, and this isn't a gentian.  

By resizing the image when I got to my desktop, I got a much better answer: 

This is correct--it's a Primrose, Latin name: Primula.  It's a genus of herbaceous flowering plants in the family Primulaceae. 

I thought I'd try asking our favorite AIs what they thought.  My prompt after uploading the image was [describe what's in this image] 

Again, a reasonable answer from Gemini, but not great.  This looks a bit like Polemonium pulcherrimum, but it's not quite right.  Identifying small blue flowers is always hard.  

By contrast, ChatGPT gives a nice description, but no identification: 

Not quite what I was looking for.... so I changed my prompt to:  [what kind of flowers are these] NOW I got a much better answer.  Here's Gemini's answer: 

And ChatGPT's: 

Which is also exactly right.  But, as always... double check.  Never trust an AI's identification of flowers, trees, dogs, or exotic symbols found on the packages of yummy chocolate!  

SearchResearch Lessons 

I didn't mean to write quite as much as I did--but this is a never-ending topic.  Let's summarize what we learned from this little escapade... 

1.  Regular Search-By-Image is great at some things, but not at others. Usually it's simple to tell when SBI doesn't work--the hits don't tell you anything about the object of the search.  BUT... you can improve your SBI query strategy by... 

2. Include location information about the object of your SBI search. We saw that including the city name vastly improved SBI search on Google.  (But, oddly, it did nothing to help Bing's SBI!)  

3. Including location information VASTLY improves AI search.  We saw that in both Gemini and ChatGPT--their results got much better when it knew where the target was from.  Kudos to Gemini for mentioning that "including the location" would help.

4. Use AI/LLMs for search at your peril.   We've talked about this before--they give what look like very plausible answers, but they can be complete nonsense.  It's tempting to believe them, but the stories AIs told me about the connection between chocolate and the baselstab symbol were truly inventive...and wrong.  

5. Flowers are still hard for AIs.  Many flowers look a lot alike (e.g., blue primroses and gentian).  I'm not really blaming AI for getting it wrong, but at the same time, they don't know their limits.  Really.  An AI SHOULD be giving you warning signals when it really can't accurately identify something like a flower or a carved symbol.  But they don't.  (And, for heaven's sake, do NOT trust an AI to correctly identify a mushroom for you.)

6. AIs are sensitive to the prompts you give them.  We found that asking an AI to [describe this image] doesn't work--be direct and say what you really want to know--e.g., [identify the flower in this image]      

So, to answer our original question, "What works?"  There isn't any one answer, but knowing what your tools can do... and what they can't do.  Be specific in your questions to an AI, provide as much information as you need, and remember your ordinary online search skills are still invaluable.  

Keep searching! 

A note about Heterophylly in the real world

Our Challenge about leaf variation on an individual plant made me more observant in the real world... 

Holly leaf variation on the same plant. Taken in Zürich by Dan.  The left image is from 3 meters above the ground in an unpruned part of the plant. The right image, from a well-pruned part of the bush, is about 1 meter off the ground.  

... I hope that being a Regular Reader of SRS does the same for you as well. 

I was out walking last week, not long after having written about Mimicry in Plants (March 14, 2025), when I happened to cross paths with a beautiful holly bush.  Like most holly bushes, it had the usual assortment of glossy green prickly leaves--it was lovely, but not the kind of bush you'd reach out and touch.  

But having just read/written about heterophylly, I took a closer look--sure enough, just as it says in the books, there was a distinct variation between the lower branches (which had been pruned extensively) and the upper branches (which had never been pruned).  Even better--the right side of the bush had a slight overhang into a neighbor's yard.  They clearly had been pruning there--on the right was an armada of sharp-tipped leaves. 

However, on the upper branches, especially on the left (and unpruned) side, smooth edge leaves abounded.  

As I wrote last week, the mechanism of holly heterophylly is pretty well understood.  The prickly/smooth variations are a result of damage to the leaves.  When the leaves are damaged (say, by a passing hungry deer or by a neighbor’s pruning shears), methylation of the DNA in the leaves happens as a result of tissue damage.  (Side note: methylation is the process of adding methyl groups onto pieces of large molecules, like DNA, to modify their behavior. This is the way much of epigenetics works. When an animal chomps on a leaf, methylation happens.)

By comparing the DNA of prickly leaves vs. smooth leaves, it turns out the prickly ones were significantly less methylated than prickless leaves, suggesting that methylation changes are ultimately responsible for leaf shape changes.  More methylation = more prickly leaves.  What's more, the methylation has an effect on nearby leaves.  Other holly leaves nearby will also develop the prickles, with the effect diminishing with distance.

I was delighted to see heterophylly in action while just strolling down the street.

Keep looking... and searching.

SearchResearch Challenge (3/25/25): To search, or to AI? That is the question...

Looking for an answer? 

Online search can often get to an accurate, trustworthy answer quickly.  But as you know, it really helps if you know a bit about what works.  

One of the major tools has been Search-by-Image, through either Google, Bing, Tineye, or Yandex.  

But now multimodal AI systems can analyze an image and tell you the story... or can they? 

As a skilled and literate searcher, you should understand the differences.  So let's give this Challenge a try to see what works, what doesn't work, and how you can tell the difference.  

I was looking for a couple things the other day while traveling through Switzerland.  When you try to find out the story of these images, where you able to use "regular" Search-by-Image, or did you use AI image descriptions?  If so, what prompt did you use, and how well did it work for you? 

1. While traveling in northwest Switzerland, I saw this emblem all over the city I was visiting.  Can you tell me what city I was in?  Just as importantly, what IS this thing? 

On a box of chocolates... 

Carved into a stone wall

Seen on a parade float

2. In open green spaces all over Zürich I keep seeing this flower popping up--they're growing everywhere in vast numbers.  What is it?  Again, how do you know?  What search method did you use? 

A few flowers carpeting the lawn. What are they?

The point of this week's Challenge is that I've had mixed results with both kinds of image searches.  When you do searches like this, what works for you?  Can you tell us HOW you got to the correct answer? 

Lots more data next week.  In the meantime... 

Keep searching! 

The road to 5 million blog views on SearchResearch!

 Without much notice... 

SearchResearch Overview, as imagined by Gemini

... SearchResearch just passed a major milestone.  We are now officially well over 5 million blog views!  (The actual number today is 5.4 million--I somehow missed the last 400,000 views by not paying attention.)

If you remember, back in mid-December 2015, we crossed over 2 million views.  

I started the blog on January 30th, 2010 (the very first post) and we quickly became a community of interested searchers sharing tips about the obvious and the (incredibly) obscure.  (In the Obscure Hall of Fame: How flowers rotate--March 25, 2010; Zouave uniforms in the Civil War--October 27, 2011; too many others to list here.)  

And, as I'd hoped, I rewrote several of my favorite posts into a book, The Joy of Search (now available in paperback).  It was a real joy to see my book in airports: 

At San Francisco airport

It was also wonderful to be able to visit bookstores and find it on the shelves: 

Found in Kramer's book shop in Washington DC

Or even MORE fun, to visit bookstores and libraries (including the Library of Congress) to speak about The Joy of Search.  Every time I spoke I mentioned our SRS blog community and how incredibly wonderful the experience has been.  

The announcement for my book talk at Books Inc. Thanks, folks!  

In a very real sense, congratulations to you all.  Without your devoted readership, 5.4 million views would not have been possible. 

As you know, I'm working on a new book (working title: "Unanticipated Consequences").  If you want to follow along in that work, subscribe to my Unanticipated Consequences substack.  Maybe I'll get the book out this year.  When I do, you'll be the first to hear about it right here in SearchResearch.  

Forward, to 10 million views!   

Hasta Luego from the SearchResearch Rancho where I'm taking the weekend off to celebrate.  (And work on my book...)  

Keep searching.  

Another view of the SRS Rancho as envisioned by Gemini. Imagine I'm relaxing here.

Answer: Mimicry in plants?

It's a simple question... 

The question this week was pretty straightforward:  

And you probably also know about some insects that mimic plants: 

 

Leaf insect. P/C Wikipedia.

Mimicry is a fairly common trick in the world of living things--mussels mimic fish, flies mimic spiders, fish mimic their environment... it continues: walking stick insects mimic sticks, mussels can mimic small fish, and Viceroy butterflies mimic Monarch butterflies. This is all well known. But..

1. Can a plant mimic another plant?  Can you find an example of one plant that does this? 

Somewhat unexpectedly, the simple query: 

     [ plant that mimics another plant ] 

teaches us that Boquila trifoliolata, a shrub common in the rain forests of Chile (and much of South America) can somehow mimic the leaves of the plant that it grows on.   It's also called the pilpil, producing edible fruit and stems that can be used for making rope.  

P/C Wikimedia image of B. trifolioata vine mimicking the leaves of the host plant

But knowing THAT leads immediately to a much harder question:  

2. How does the mimicking plant come to be a mimic?  What’s the mechanism by which Plant A comes to look like Plant B?  

This is a bit of a mystery. The pilpil is the only plant species reported to engage not JUST in mimickery, by also in in mimetic polymorphism. That is, the ability to mimic multiple host species simultaneously. This obviously doesn't happen in animals--each animal mimics only one other animal at a time. But somehow, the pilpil manages to mimic multiple species at once.

As Wikipedia tells us, this is a form of Batesian mimicry, when a harmless species mimics a harmful one to ward off predators.

But how does it do the mimicking?

There are a lot of hypotheses about the mechanism include (e.g., microbially mediated horizontal gene transfer, volatile organic compound sensing, and the use of eye-like structures), but nothing seems to have panned out.

On the other hand, looking for:

[ leaf variation on single plant ]

leads us to learn about two concepts new to me: heteroblasty and heterophylly.

Heteroblasty is a significant and abrupt change in form and function, that occurs over the lifespan of certain plants. Like the pilpil changing leaf shape to match the host plant.

Heterophylly is when a plant has multiple leaf shapes on a single plant due to its environment.

As an example, Sassafras (Sassafras albidum) is well-known for having three distinct leaf shapes on the same plant - oval (unlobed), mitten-shaped (two-lobed), and three-lobed all on the same plant. It's the best known example of heterophylly.

More relevant to our discussion, holly leaves (Ilex aquifolium) can also make different types of leaves at the same time, even on the same branch--some with prickles, others without.

P/C after Herrera 

But the mechanism of holly heterophylly is pretty well understood. The prickly variations are a result of deer eating the leaves of the plant. When the leaves are damaged (say, by a passing hungry deer), methylation of the DNA in the leaves happens as a result of tissue damage. (Side note: methylation is the process of adding methyl groups onto pieces of large molecules, like DNA, to modify their behavior. This is the way much of epigenetics works. When an animal chomps on a leaf, methylation happens.)

By comparing the DNA of prickly leaves vs. smooth leaves, it turns out the prickly ones were significantly less methylated than prickless leaves, suggesting that methylation changes are ultimately responsible for leaf shape changes. More methylation = more prickly leaves. What's more, the methylation has an effect on nearby leaves. Other holly leaves nearby will also develop the prickles, with the effect diminishing with distance.

While the variation in leaf shape has been known for a while, it’s now clear that changes in leaf type are associated with differences in DNA methylation patterns, that is, epigenetic changes do not depend on changes in the sequence of DNA, but result from trauma to the plant.

What does this mean for our friend the shape-shifting pilpil? It demonstrates that changes to leaf shape can be epigenetic (that is, the plant doesn't have to modify its DNA, but just tack on a few extra methyl groups here and there).

That doesn't fully explain the way that pilpil leaves can mimic the host plant, but it does suggest a mechanism for changing the leaf structure.

Another intriguing hypothesis is that there is some kind of "visual sensing" that's going on with the vine. What makes this idea particularly interesting is that Boquila can mimic different hosts on the same vine without direct contact with the model leaves, suggesting some form of distance sensing. If the vine is truly using visual cues, it would be amazing--and completely novel. The big problem here is that nobody seems to be able to (pardon the pun) see any such organs!

A more probable hypothesis is that there is some kind of individual plant recognition, perhaps by sensing the release of nearby volatile organic compounds from recognized plants.

Kudu grazing on an acacia tree, causing the tree to put out a cloud of ethylene, telling other
nearby acacias that the browsers are here--increase your tannin load.  

Acacia trees, for instance, can detect ethylene emissions from neighboring damaged trees, triggering increased tannin production in the leaves of the acacia tree as defense against grazing kudu. [Heil, 2010] Other plants do similar things: Arabidopsis thaliana (a small plant in the mustard family) can also detect volatile compounds like methyl jasmonate from neighboring plants that are injured, which triggers its own defensive responses.

So it's not much of a leap to imagine that as a Boquila trifoliolata vine grows from tree to tree, each part of the plant might sense a different host that it's growing on, and invoke different responses--on each different part of the vine--depending on what chemical signals that part of the vine senses. It's also true that mimetic changes appear to be very localized, primarily affecting the leaves within 60 centimeters of the host plant. That's perfect chemical sensing range.

This effect would be mimetic polymorphism at a very fine level of detail.

To broaden my search I asked Claude for:

[ any plant that grows differentially depending on the chemical signals it senses]

I learned about the Centaurea maculosa (spotted knapweed) that detects specific root compounds from competing plants, responding by increasing production of allelopathic compounds (deadly poisons for the competition), essentially tailoring its chemical warfare based on which neighbor it detects.

Obviously, I did a Google Scholar search to verify that claim, and found a wonderfully detailed paper [Kong, et al, 2024] that goes into great detail about how the spotted knapweed senses the competition and then emits specific poisons to kill off the competition!

Just as obviously, I don't know if this hypothesis is correct--we need a good field botanist to do some studies, but it's not crazy.  All of the mechanisms are there and could be the product of evolution.  (And is very similar to the mechanism proposed by [Gianoli, 2014].)  

It's remarkable what you can learn (and hypothesize about) with some desk research!   

SearchResearch Lessons 

1. As with most complex searches, you have to learn as you go.  Note the new terms we had to learn to answer this question (mimetic polymorphism , methylation, heteroblasty, heterophylly).  Learn as you go in order to get more deeply into the topic.  

2. Interleaving "regular search" with LLMs (e.g., Claude, Perplexity, ChatGPT, etc.) can be really useful.  I was able to learn new terms and concepts by working with the AIs.  As always, be sure to CHECK their work. It's like reading an unreliable narrator in a novel--they're useful, but can't be trusted.  

 

Keep searching!

----

Citations:

Gianoli, E., & Carrasco-Urra, F. (2014). Leaf mimicry in a climbing plant protects against herbivory. Curr Biol, 24(9), 984-987.

Heil, Martin, and Richard Karban. "Explaining evolution of plant communication by airborne signals." Trends in ecology & evolution 25.3 (2010): 137-144.

Herrera, C. M., & Bazaga, P. (2013). Epigenetic correlates of plant phenotypic plasticity: DNA methylation differs between prickly and nonprickly leaves in heterophyllous Ilex aquifolium (Aquifoliaceae) trees. Botanical Journal of the Linnean Society, 171(3), 441-452.

Kong, C. H., Li, Z., Li, F. L., Xia, X. X., & Wang, P. (2024). Chemically mediated plant–plant interactions: Allelopathy and allelobiosis. Plants, 13(5), 626.