Answer: What building is this?

There's an important cautionary tale this week.  AIs sometimes get it wrong... especially with image search...

A building in downtown Denver. P/C Dan Russell 

Last week we asked a relatively simple question:  What building is this?  (See above.) 

If you do a regular old Google Image Search, you get the right answer:  This is the El Jebel Shrine, aka the Sherman Event Center in Denver, Colorado. That’s great, and exactly what you’d expect.  Easy peasy.

If you search on Bing Image Search, you also get the right answer: 

But I wouldn’t write an SRS post about something so obvious. 

What IS surprising is what happens when you ask your favorite AI / LLMs about this image.  

Mostly, they get it very, very wrong.

Oddly, after getting it right with Bing Image Search, Microsoft's Copilot gives a terrible answer.  (Once again, the left hand does not know what the right hand is doing.)  

 

This is NOT the Denver Athletic Club.  (It is also made of brick and has arches, but no domes or minaret-like towers.)  

But if you use the visual description ability of ChatGPT, you get another very wrong answer (mostly because it’s trying to find buildings near my Palo Alto, California location–an assumption that seems really bad… especially since none of the buildings it suggests look anything like the image I asked about! 

I thought that maybe I should give ChatGPT a hint, telling it that the building was in Denver. But that didn't work either. In fact, the answers got worse. The buildings it suggested aren't anywhere near the search target!

Maybe I'm just asking the wrong LLM?

Here's Claude's reply:

Claude gets the style correct, but not a proper identification.

I then asked Google Gemini what the answer might be.  AGAIN:  It too is really wrong: 

I know the Advanced Medicine Center Building in Palo Alto--it looks nothing like this.

Well, I thought, that’s because I’m using Gemini 1.5 Pro.  (As you know, there are multiple Gemini models to choose from...)

But when I switched to Gemini 2.0 Flash Experimental (the latest!), I got an even more wrong answer, even though it has a “might not work as expected” disclaimer.  Indeed!  

The Mosque of Ibn Tulun looks a little like the image with domes and towers, but the color, layout, and materials are all wrong.  

HOWEVER, when I tried Gemini 2.0 Experimental Advanced, I finally got the correct answer. 

Notice that it took 3 different tries with different Gemini models to get to the right answer.  That’s not encouraging.  We know that most people will simply accept the first result and not do any follow-on checking.  

Since I took the photo I’ll tell you: this really IS the Sherman Street Event Center, aka the El Jebel Shrine, aka the Rocky Mountain Consistory, and as the Scottish Rite Temple is a historic building in the North Capitol Hill neighborhood of downtown Denver. 

Here’s a great article about it: https://denverite.com/2024/05/09/el-jebel-shriner-mosque-photos/ 

The Moorish-inspired building was constructed in 1907, as a meeting hall for the El Jabel chapter of the Ancient Arabic Order of the Nobles of the Mystic Shrine (the Shriners). It was never a true mosque in the Islamic sense. In 1924, having outgrown the building, the Shriners sold it to the Scottish Rite Masons, who renamed it. In 1995, the Scottish Rite sold the building to Eulipions, Inc. who converted it into a catering and events facility, and it’s been bouncing around the Denver real estate market ever since.  (Although it seems to have recently landed a permanent owner.) 

This is a beautiful example of the Moorish Revival Architecture movement of the early 20th century in the US. This stylistic movement is a variation of Islamic architecture that was introduced in the 17th and 18th centuries in Spain. It's characterized by geometric shapes, arches, and decorative elements like arabesques and ceramic tiles.

Bottom line: The AIs are mostly wrong.  Regular search-by-image is much better.  (Oddly, Tineye.com found nothing, not even a near miss!)  

But what’s worse is that they’re CONFIDENTLY wrong.  There’s no hesitation, no questioning of the plausibility of the results.  When Microsoft CoPilot says that this is the “Denver Athletic Club,” it doesn’t say that it “might be” the club, or that “this image looks a great deal like the Athletic Club, but I’m not 100% sure.”  

Which is disappointing.  

Search Research Lessons

1. Don't trust any image identification that the LLMs give you. We've talked about this before, but REALLY... if they can't identify a very visually distinct building, I wouldn't trust their results when foraging for mushrooms, berries, or edible plants! Perhaps one day they'll improve their accuracy... but for the time being, be sure to double-check everything!

As always,

Keep Searching!

SearchResearch Challenge (1/22/25): What building is this?

 This should be a simple question... 

P/C by Dan Russell

... but it turns out that identifying this building is either really simple... or really hard... it all depends on which tools you use!  

This leads to our SRS Challenge today: 

1.  What is the name of this building and where is it?  What style is it?  (For extra credit--why was it made in this rather elaborate style?  

It shouldn't take you long to find the CORRECT answer, but many of the tools out there in use today are giving really terrible responses.  Can you filter out the wheat from the chaff?  

Let us know what you find, and how you know that it's the right answer!

Keep searching!  

Answer: Earliest photograph from Cozumel, Mexico?

 I know, I know... 

Lionfish at 20m (60 feet) on the reef in Cozumel, Mexico

 

This is the Nth time we've looked for the "first photo of X"--but I noticed something very different in this Challenge. As we've noticed before, even very simple questions sometimes bring up unexpected surprises and issues.  That's the case with this week's very simple question. 

I started searching for the earliest photos I could find taken in Cozumel.  Easy, right?  Turns out there are interesting issues here.  See if you can spot the biggest issue and work around it! 

This week's SearchResearch Challenge isn't hard, but I think it's an important one that speaks to the future of doing online research.  Let's figure it out together.  

1. What is the earliest / oldest photograph you can find that was taken on Cozumel?

If you do the obvious query, this is what you'll see on Google: 

That's pretty disappointing. It's a really interesting idea, but I am very skeptical of the claim that these Mayan "images" are from projections of sunlight onto the limestone walls in Cozumel.  To quote from the summary: 

"The discovery suggests that the Mayans were able to project images and record them on stone tablets."

What?  Really?  That's a wild claim.  So I thought I'd try Bing search--only to find this: 

What is going on here?  Did the Mayans really invent a precursor to photography 1000 years ago?  

A little bit of digging into the links below each of these results reveals that the SOURCE of these remarkable claims is this page from "This is Cozumel" tourist web site. 

Check out the date: It's April 1, 2014.  The whole article is an April Fool's joke. 

As we know, search engines and LLMs are notoriously bad at humor and satire.  (See this article "How satire crippled Google's most powerful AI.")  And this is a great example of the LLMs not understanding humor, but presenting it as fact.  Both Google and Bing got it terribly wrong.  

I then tried Gemini, and got a better answer: 

This is actually pretty interesting.  For fun, I also tried this query on Claude: 

(I tried Perplexity as well and got a similar result. This is a good sign.) 

The most interesting thing here is the mention of Augustus Le Plongeon and his wife, Alice, who visited the Yucatán and Cozumel in the 1870s. Now THAT is a useful lead for us SRSers. (It's worth reading their Wikipedia page. What an adventurous couple!)  

I did the obvious searches, e.g. this one and similar:  

     [ Le Plongeon Cozumel photographs OR photo ]

and quickly learned that the best resource for this Challenge were the books that they'd published (or were written about them).  

As you know, Google Books is a great resource, but once again the Internet Archive (Archive.org) is a fabulous resource:

Notice that I did a search on Archive.org searching the full text (and not just the metadata).  This led me to a happy morning of reading several books that were published about the Le Plongeons travels in Mexico.  

Example: I read through Lawrence Desmond's book, Yucatán through Her Eyes: Alice Dixon Le Plongeon,Writer & Expeditionary Photographer. Albuquerque: University of New Mexico Press, 2009.  In there I found that:

“In early February, [1877] they decided to leave Isla Mujeres and sail for Cozumel Island, about seventy-five miles south of Isla Mujeres and ten miles off the coast. Today the island is a tourist destination complete with an international airport, but in the 1870s only a few travelers visited the island, and Alice noted it had only “five hundred inhabitants” (ADLP 1886a:28).”  Page 222. 

In that book, on page 90, you'll find a 1876 photo of several large axes and flint spear heads on page 90.  So they're clearly taking photos on Cozumel in 1876.  

Then, in the book: The Mayas, the sources of their history. Dr. Le Plongeon in Yucatan, his account of discoveries, by Salisbury, Stephen  you'll find several letters from Le Plongeon in 1877, written from Cozumel and commenting that they left Cozumel in 1879.   

Finally, after lots of scanning around through various books, I landed on Here and there in the Yucatan Alice D. Le Plongeon. (1889)  On page 36 you'll find this photo: 

P/C Wikimedia. From the book "Here and There in the Yucatan" by Alice Le Plongeon.

The book was published in 1889, but the photo was probably taken in 1876 or 1877 when the Le Plongeons were visiting Cozumel.  

If you remember from earlier discussions, photography was invented in 1822, and the first real photos started coming out in 1826. 

So I'm pretty sure this is the one of the first photos taken on Cozumel.  

SearchResearch Lessons

1. Don't believe everything you read.  LLMs can be (as we've seen multiple times) misled by all kinds of things, including humor.  If you see something extraordinary, check it out in detail.  

2. Look at what you find and use names of people / places as leads.  Here I used the Le Plongeons as a way to find the earliest photos of Cozumel.  (I admit that I got lucky here.  But the point remains: leverage what you learn.)  

3. Books can be immensely valuable!  That might seem obvious, but I often talk with searchers who give up because they can't find a web page with the information they seek.   

4. Use Archive.org for your book research.  It’s really worthwhile getting an account at Archive.org so you can read the full text (with images) of the books you seek.  

Keep searching! 

SearchResearch Challenge (1/8/25): Earliest photograph from Cozumel, Mexico?

Simple questions are sometimes tricky. 

Lionfish at 20m (60 feet) on the reef in Cozumel, Mexico

As we've noticed before, even very simple questions sometimes bring up unexpected surprises and issues.  That's the case with this week's very simple question. 

As it turns out, I'm taking a vacation week in Cozumel, Mexico.  I'm scuba diving with my buddies and as you know, travel often begets questions in the curious mind. 

I started searching for the earliest photos I could find taken on Cozumel.  Easy, right?  Turns out there are interesting issues here.  See if you can spot the biggest issue and work around it! 

This week's SearchResearch Challenge isn't hard, but I think it's an important one that speaks to the future of doing online research.  Let's figure it out together.  

1. What is the earliest / oldest photograph you can find that was taken on Cozumel?

I know, it sounds simple--but can you find the issue with this question?  

Let us know what you find. 

Keep searching! 

 

Imagine that I'm here.
P/C LRM from Pexels.com

Answer: What's the most significant thing going on here? (3/3)

I didn't mean to write three posts...  

But this is a big problem!     

I have to sympathize with Ramón and remmij’s comments about having a sense of overwhelmment in the SRS space.  We’re living in a frenzied time–when new systems and products are bubbling up every day and in every way.  It’s easy to give up… but don’t.   

A Gemini-generated image of Dan being overwhelmed by the
number of choices when doing Deep Research.

Here, at the SearchResearch Rancho, we’ll try to shed some light on what works, and what you can safely ignore. Isn’t that why you come here?  For a bit of clarity and guidance in these complicated times?  Let’s see what we can figure out.  

As I mentioned earlier this week, a new trend in SRS is the launch of several new “deep research” tools.  In today’s post I’m going to compare / contrast three of them.  (For simplicity, and to make a small pun, I’m going to call these tools “DR” tools.   

(1) Google’s DR tool is their “Deep Research” mode for Gemini.  (I can’t give a direct link to it because Gemini doesn’t use URL parameters.  You have to manually select the “1.5 Pro with Deep Research” option to get it to show up.  Sorry about that.)  The idea is that "1.5 Pro with Deep Research" will write you a short summary on some research topic.  

(2) Undermind.ai  a "personal assistant" to help you with collecting, analyzing, summarizing work in a particular topic area.  

(3) OpenScholar (from Allen.ai) a DR tool that scans the scholarly literature (and sometimes it refuses to do so!) and writes a summary much in the style of Google 1.5 Pro w/ DR.

(For a list of other deep research systems, take a look at the SlashDot list of DR systems.) 

The idea behind all of these DR systems is to use AI techniques to analyze large volumes of complex data, looking for in-depth insights and discovery beyond traditional research methods. The hope is that these tools will find complex relationships and patterns within the data that might be difficult to identify manually.  Each of them writes a little report on what they found giving citations to the literature they used. 

To compare each of these systems I asked two questions.  

A. What has been the effects of the creation of Lake Nasser on the ecosystems around it?

B. What has been the effect of the creation of Lake Nasser on the incidence of schistosomiasis in Egypt? 

Question A is to answer our primary question (“what are the most important changes..”) 

Question B dives more deeply into a particular question about schistosomiasis, a parasitic disease that’s caused by blood flukes (trematode worms) of the genus Schistosoma transmitted by freshwater snails that often live in agricultural canals.. It’s a very serious disease. Can these DR systems help us understand this important change?   

Here’s what I found when using these DR systems… 

Question A:  (effect of Lake Nasser on the ecosystems)  

Google:  The DR tool (using Google’s “Gemini Advanced 1.5 Pro with Deep Research” on January 2, 2025) created a 2400 word report (link to report) that covers a bit of the history of Lake Nasser, and then lists positive and negative effects. Ecological projects are mentioned and a section entitled “Scholarly Research on the Ecological Impact of Lake Nasser” includes a very obscure data set on water quality.  (The data is important–but why copy/paste part of the raw data in the report?)  It mentions the “South Valley Project” (another name for the Toshka Lakes). 

Oddly, the report cites TWO Kids.Brittanica.com reports, a few studies by international organizations (e.g., WorldFish international), a few Wikipedia articles, and a couple of scientific literature studies.  

Overall grade: It’s not bad, but there are a few sections that are just odd–not something a human would ever write.  

=================================

Undermind:  When you use Undermind, it starts with a little back-and-forth trying to get you to add more detail to the research question. That’s fine, but it also ends up narrowing the scope of the research. In my case, the final question posed to Undermind was “The ecological impacts of the creation of Lake Nasser, focusing on changes and adaptations in both aquatic ecosystems within the lake and terrestrial ecosystems surrounding the area.”  

Here’s the top of the Undermind report: 

Undermind also provides a helpful summary of the Categories of papers it found: 

And a very helpful timeline of research work: 

As well as a very interesting set of clusters of research groups and contributions by each group: 

The references (at the end of the report show the expected citation info, but also a measure of the topic match, the number of citations / year (indicates how often it gets cited), and a summary of the relevance of the paper to the topic: 

Overall grade: Undermind gives you much more information that Google’s DR report, including analysis that Google won’t give you.  (e.g., the clustering) 

=================================

OpenScholar: By contrast, Ai2 OpenScholar took the same research question and wrote a fairly short, cursory report. Here’s the top of that report: 

There are only 3 references given, and one of them (Goher et al. 2021) is used for 6 of the 10 citations. What’s more, the Goher paper was published in the journal Water, which is a publication of the MDPI (Multidisciplinary Digital Publishing Institute), which has a not-great reputation. (You can read the Wikipedia page to learn more.)  In any case, it’s not a paper I would choose to center my critical review of Lake Nasser ecosystems, even though the paper data seems reasonable enough.  

Overall grade: It’s not bad, but there are a few sections that are just odd–not something a human would ever write.  And the overall quality of the cited works was a little suspicious.  (I found other examples of papers I’m not sure I would cite in OpenScholar analysis reports.  OpenScholar–what are you doing??)  

=================================

Question B: (effect of Lake Nasser on Schistosomiasis)

Google:  In reply to the Schistosomiasis research question, Google’s DR tool created another report (link to report) that is structurally similar to the other report… it too covers a bit of the history of Lake Nasser and environmental issues.  It has a section on public health interventions, and then the effects of Lake Nasser on the incidence of Schistosomiasis, pointing out that a massive anti-snail / anti-Schistosomiasis campaign has caused an overall REDUCTION in the incidence of the disease.  

Overall grade: Again, it’s not bad.. But there are some contradictory statements (e.g., while Schistosomiasis overall has gone down, another subspecies of Schistosomiasis (specifically, Schistosomiasis mansoni) has actually increased.  The obvious question a human would ask is “is this overall good, or bad for the country.”  That’s never really addressed.  

=================================

Undermind:  By contrast, Undermind doesn’t really write much of a report–it really gives a bunch of research result in the literature.  What little it says about the incidence of Schistosomiasis slightly contradicts Google.  In particular, it writes that: 

  “The creation of Lake Nasser after the construction of the Aswan High Dam significantly increased schistosomiasis transmission in Upper and Middle Egypt by altering ecological conditions that favored the proliferation of snail vectors …, though public health interventions such as mass drug administration and mollusciciding effectively reduced disease prevalence in most areas despite persistent hotspots.”  

Overall grade:  Undermind gives a completely different set of relevant papers than Google Gemini!  (There is zero overlap.)  But it does give a deeper analysis about why Lake Nasser changed the way that Egyptians use canals for agriculture, leading to an increase in the disease!  

=================================

OpenScholar: I’m not sure what happened here, but I was completely unable to get it to give me any kind of answer to my schistosomiasis questions!  I tried multiple variations on the research statement question, but all I ever got was a failure notification.  

“Referenced task failed. Error: We were unable to retrieve any relevant papers for your query. Please try a different query. OpenScholar is not designed to answer non-scientific questions or questions that require sources outside the scientific literature.” 

Overall grade: Not a great performance.  I spent probably an hour trying different variations on the theme (including logging in on a different account), but for whatever reason, it just refused to answer.  

SearchResearch Lessons

Let’s start with the post from 2 weeks ago… 

1. When getting an overview, consider using maps… and in particular, consider using time lapse.  They’re relatively easy to get, and give you a very different perspective on regional questions.  

2. Asking LLMs these questions is a good idea.. BUT ONLY if you look at multiple different AI systems.  As we saw last week, each of these can give you a very different idea about what the issues are.  But aggregating the results can give you a decent overview.  (Do not, however, take the frequency of topics mentioned as a proxy for importance!  That’s kind of random–multiple mentions do not equal overall importance!)  

3. The Deep Research (DR) tools are a new kind of thing for doing serious research. While interesting, they’re not a substitute for real human research.  (At least not yet.)  Like LLMs, each has their own perspectives (which can be useful when taken together), but don’t seem to have great quality control about what papers are high quality.  

As ever, check your work--now more than ever.  The DR systems are really interesting tools, but they’re not quite a replacement for your good research skills and discernment.  

In the future we’ll talk about other DR tools, including: ResearchRabbit, Elicit, Iris, Affor.ai and NotebookLM.  We’ll talk about those soon.  (But I didn’t want to write a book about them.. At least not yet!)   

Keep searching!