SearchResearch (12/31/25): Review of the year at the SearchResearch Rancho

 It's the day of the year when we look back... 

... at what we've done and review all of the accomplishments (and potentially any issues that might have cropped up). 

Naturally, I thought I'd look over all 55 posts from 2025 and see what themes emerged.  I did that, and came up with a few biggies (this list is from the top of my head): 

1. Shift the focus from purely-Challenge based to more commentary on research methods.  (Oct 24, 2025)

2. Methods for doing just-in-time research. (Aug 15, 2025) 

3. Fun Challenges: such as Mysteries from Zürich. (July 2, 2025) or Lemon-in-Lemon (April 24, 2025)

4. Comments about the quality of doing AI research. (May 13, 2025)  Or using AI for creating summaries. (May 15, 2025)

It's pretty clear that the old methods of doing online research are profoundly changing.  Take a look at my friend Daniel Tunkelang's recent post about Is Retrieval Dead?  In that post he muses on how people now expect search to return answers and not just a list of links.  

But as he points out, 

"This is how answer engines fail in practice, not with absurd hallucinations but with reasonable yet dangerously incomplete narratives. Retrieval would have surfaced vetted medical guidance, differential diagnoses, and clear warning signs. It reveals the shape of the problem rather than collapsing it into a comfortable, concise, but incomplete picture." 

I think one of the points of this blog is to highlight the changing nature of online research.  

Regular (classic) search shines in those areas where researchers need to understand and navigate tradeoffs.  And, to Daniel's point, we still need to know something about the answers we're given by AI answer engines, they often remove context and make errors... and that's why you need to constantly check what you get.  

Case in point--me trying to make a summary using AI: 

I thought that I'd create an AI summary that would crawl through all of the posts from SRS this year and create a good synthesis of what we've covered.  

But if you remember our previous attempts to do this, it often ends in compromises.   

However.... AI quality is improving, and I'm an optimist, so I wanted to try again.  

So... I got a list of all 55 posts from my blog.  (Here's the accurate, master list of all 55 posts that I got straight out of my blogging platform, Blogger.)  

I put those into a Google Sheet (here) and asked Gemini to give summaries of each post in Column H. 

Here's my prompt after uploading my spreadsheet:  

[this is a spreadsheet where each row is one blog post title and the URL to the web page for that post.  Please create a new spreadsheet that has the title of the blog post, the URL of the blog post, and a 100 word (or less) summary of that blog post]  

This is where things start to go wrong. I looked at the output: Argh!  This is a terrible result (click thru on the image to see in full size). What's going on?  

I hate debugging AI systems like this.  But here goes... 

The summary of the first blog post (row 1) is pretty good "It analyzes the variable accuracy of AI (like Gemini) in performing OCR on images of bookshelves, finding that it "kinda/sorta works" with up to 95% accuracy, but results are inconsistent, require validation, and improve when users request data in a spreadsheet format."  That's nice.  

But most of the other results are terrible.  What's going on? Why so many errors? Why can't it find the web page from that URL? Row 20 is from outer space.

The short answer: Gemini incorrectly extracts the URL from the link and that leads to a cascade of failures.

For example, in row 17 is the URL: 

https://searchresearch1.blogspot.com/2025/09/coffee-sniffers-video.html 

You know what's odd about that?  That's not the right URL!  Here's the ACTUAL URL to my post: 

https://searchresearch1.blogspot.com/2025/09/a-short-video-about-coffee-sniffers.html 

See the difference?  The page name in the original is "a-short-video-about-coffee-sniffers.html" 

Here's what the original link looks like: 

         A short video about the Coffee Sniffers (a test vi...

If you or I were asked to extract the URL from that text link, we'd get the correct URL. 

But somehow, when I asked Gemini to extract the URL from the link text, Gemini munged it to be: "coffee-sniffers-video.html" -- and that's not a real page, which is why Gemini can't give a decent answer in Column H. 

The truly strange thing here is that SOMETIMES the extraction works properly (see row 1 or row 5, both of which work beautifully).  But mostly it messes up, changing dates around, dropping a few characters here and there.  It looks to me as though the AI is trying to do spell correction, but it doesn't know that spell correction in a URL is senseless.  

I fooled around with this for hours trying different approaches to make it work.  I failed at all of them. 

Finally, I tried using ChatGPT... and found similar kinds of results. (See my ChatGPT spreasheet for yourself.)  I have to admit that it's much better than the Gemini output.  Different errors, a few occasional successes, but about 50/50 good results mixed with reasonable sounding nonsense.  Overall, another multi-hour waste of time.  

Blarg.  

What to do now?  

A Path to Success by Using Claude:  

Then, in desperation I tried using Claude--and it worked beautifully.  

Using exactly the same approach (that is, the same spreadsheet as input with the same prompt as above), Claude figured out how to correctly extract the URLs from the links, and then correctly created 55 summaries, one for each blog entry.  You can see the results here: Spreadsheet of links, URLs, and summaries created by Claude.  The resulting spreadsheet looks like this: 

That's much better.  

Having learned my lesson, I then hand-checked each of the Summaries, making sure they were an actual summary of the linked blog post AND that they were a decent summary.  Here's a nice example summarizing the blog post Nov 5, 2025, "Pro tips on using AI for deep research."  

Pro tips for using AI to enhance reading comprehension. Shows how to use LLMs to explain obscure phrases in P.G. Wodehouse novels by providing book context. Warns about hallucinations even in seemingly accurate explanations.

Bottom line:  AIs often look like they're doing the right thing, but on close inspection, the results are less than you want.  CHECK YOUR WORK.  

And, if you want to do this kind of AI-driven manipulation of URLs, texts, and summaries--my recommendation is to use Claude.ai -- it did a superior job without a lot of hassle.  It just worked, writing its own code, debugging it, and executing it in a solid way.  

And now, I can finally do what I wanted to do in the first place--give a summary of SearchResearch for 2025:  

Post title	Post summary
Answer: What's the most significant thing going on...	Answer about significant current events. Discusses major developments and how to research evolving news stories effectively.
SearchResearch Challenge (1/8/25): Earliest photog...	Challenge asking about the earliest photograph from Cozumel. Explores historical photography and archival research methods.
Answer: Earliest photograph from Cozumel, Mexico?	Answer about the earliest photograph from Cozumel, Mexico. Traces photographic history in the region through archival research.
SearchResearch Challenge (1/22/25): What building ...	Challenge presenting a mystery building photo. Asks readers to identify the building using various search and research methods.
Answer: What building is this?	Answer identifying a mystery building. Uses reverse image search and architectural analysis to identify the structure and its history.
SearchResearch (2/6/2025): SearchResearch, Search,...	Overview of SearchResearch, search skills, and sensemaking. Introduces the blog's themes for new readers and previews upcoming topics.
SearchResearch Commentary (2/13/25): Using Notebo...	Commentary on using NotebookLM for research. Explores capabilities and limitations for organizing and synthesizing multiple documents.
SearchResearch Challenge (2/19/25): Asking questio...	Challenge asking readers to test AI image questioning capabilities. Provides test images and asks for evaluation of different AI tools' performance.
Answer: Asking questions of images with AI?	Answer about using AI to ask questions of images. Demonstrates multimodal AI capabilities for image analysis while noting significant limitations and errors.
SearchResearch Challenge (3/6/25): Mimicry in plants?	Challenge about plant mimicry. Asks how plants can mimic other plants without eyes or nervous systems, and what mechanisms enable this adaptation.
Answer: Mimicry in plants?	Answer about plant mimicry. Explains various forms including Boquila trifoliata mimicking nearby plants, and discusses heterophylly where single plants produce different leaf shapes.
The road to 5 million blog views on SearchResearch!	Celebrates reaching 5 million blog views on SearchResearch. Reflects on the blog's impact and community over 15 years of writing about search skills.
SearchResearch Challenge (3/25/25): To search, or ...	Challenge asking when to use traditional search vs. AI chatbots. Explores the complementary nature of these research tools.
A note about Heterophylly in the real world	Note about heterophylly (different leaf shapes on same plant) observed in real-world settings, connecting to previous plant mimicry discussion.
Answer: To search, or to AI? That is the question...	Answer comparing search engines vs. AI for research tasks. Concludes both have roles: search for current info and specific facts, AI for synthesis and explanation.
SearchResearch Challenge (3/4/2025): Can you extra...	Challenge asking if AI can effectively extract and summarize an entire blog. Tests the limits of AI comprehension across many posts.
Answer: Can you extract and summarize a blog?	Answer about extracting and summarizing blog content. Demonstrates various AI and traditional methods for collecting and synthesizing blog posts into useful summaries.
Follow-up: Can you extract and summarize a blog?	Follow-up on blog extraction and summarization challenge with additional methods and reader solutions.
SearchResearch Challenge (4/17/25): What's a lemo...	Challenge asking what it's called when a lemon grows inside another lemon. Presents photos of this botanical curiosity and asks for the scientific terminology.
Answer: What's a lemon in a lemon called?	Answer about 'lemon in lemon' phenomenon. Explains it's called 'proliferation' or 'internal proliferation'—a developmental abnormality where a second fruit grows inside the first.
A lemon in lemon - update	Update on the 'lemon in lemon' post with additional reader contributions and clarifications about botanical terminology.
SearchResearch (5/1/25): How good are those AI sum...	Challenge asking readers to evaluate AI-generated summaries. Requests testing various AI tools and reporting on accuracy and usefulness.
Slightly delayed...	Brief post noting a delay in the regular blog schedule due to personal circumstances.
Special: Why are AIs so bad at diagramming?	Special post examining why AI tools struggle with diagramming tasks. Despite advances in text and image generation, creating accurate diagrams remains challenging for current AI systems.
Answer: How good are those AI summaries anyway?	Answer evaluating AI summary quality. Tests various AI tools on summarization accuracy, finding generally good results but with occasional hallucinations and missing context.
SearchResearch (5/22/25): What's the story of Rosa...	Challenge about Rosamond Lake. Asks readers to research the story behind this California lake—its name origin, history, and significance.
Answer: What's the story of Rosamond Lake?	Answer about Rosamond Lake in California's Sierra Nevada. Explains the lake's geology, naming history, and ecological significance in the region.
SearchResearch Challenge (6/25/25): Mysteries in ...	Challenge about mysterious discoveries in Zürich. Presents several research puzzles encountered while visiting Switzerland.
Answer: Mysteries in Zürich?	Answer about mysteries discovered in Zürich. Discusses finding unexpected historical artifacts and documents in Swiss archives, emphasizing serendipitous discovery in research.
SearchResearch Challenge (7/9/25): So what ARE LLM...	Challenge asking readers to evaluate LLM strengths and weaknesses. Seeks specific examples of what AI chatbots excel at and where they consistently fail.
Answer: So what ARE LLMs good at? What are they ba...	Answer evaluating what LLMs do well and poorly. Good at: summarization, translation, coding assistance, creative writing. Poor at: math, consistent facts, real-time info, citing sources accurately.
SearchResearch Challenge (7/23/25): Wake up and sm...	Challenge about the history of coffee in 19th century Germany. Asks what was strange about coffee, what 'sniffers' were, and how this affected modern coffee culture.
Answer: Wake up and smell the 19th century coffee?	Answer about 19th century coffee 'sniffers' in Prussia. Frederick the Great banned coffee roasting to protect German beer industry. Officials called Kaffeeriecher would sniff out illegal roasters.
SearchResearch Challenge (8/6/25): Best ways to do...	Challenge asking about the best AI-driven methods for quick learning. Requests readers share their experiences with micro and meso-learning techniques.
Answer: Best ways to do just-in-time learning. Par...	Part 1 of just-in-time learning series. Discusses microlearning (under 5 minutes) and mesolearning (5-60 minutes) strategies. Recommends searching for existing tutorial content as a first step.
Answer: Best ways to do just-in-time learning. Par...	Part 2 of just-in-time learning series. Shows how to use chatbots in conversation mode to have topical essays read aloud and engage in follow-up discussions on any subject.
Answer: Best ways to do just-in-time learning. Par...	Part 3 of just-in-time learning series. Demonstrates using NotebookLM to create audio/video summaries from curated research papers. Shows how to apply the Feynman technique with AI assistance.
Update: What are those bright patches of water wit...	Update on mysterious bright patches in ocean water. Identifies them as 'surface slicks'—biofilms of fatty compounds that suppress wavelets. Notes that AI tools failed to correctly identify them.
SearchResearch Challenge (9/3/25): What kind of ar...	Challenge presenting an unusual 16-sided building in Lucerne containing a massive circular painting. Asks what this art form is called and about similar installations worldwide.
A short video about the Coffee Sniffers (a test vi...	A short NotebookLM-generated video about Prussian Coffee Sniffers from the 18th century—officials who sniffed out illegal coffee roasting when Frederick the Great tried to ban coffee to boost beer sales.
Answer: What kind of art is this?	Answer identifying cycloramas as 360-degree panoramic paintings. Explains the Bourbaki Panorama in Lucerne depicting the 1870-71 internment of French troops in Switzerland, and lists US cycloramas including Gettysburg.
SearchResearch Challenge (9/17/25): New agentic wa...	Explains how to set up automated AI research summaries using ChatGPT and Gemini scheduled tasks. Provides tips on writing effective prompts for regular topic monitoring, similar to Google Alerts but AI-powered.
SearchResearch Challenge (9/24/25): What's the sto...	Challenge about greenhouses seen from aerial views. Asks about greenhouse history, what's grown under massive Chinese greenhouses, and the state of robotic/vertical greenhouse industry.
Answer: What's the story with the greenhouses?	Answer about greenhouses throughout history. Explains Roman specularia used mica or selenite, modern greenhouse origins in 17th century Netherlands, and addresses the boom/bust of vertical farming startups.
SearchResearch Challenge (10/8/25): How can the sa...	Challenge asking about the locust's different appearances. Based on Jeffrey Lockwood's book about the 1874 Locust Plague. Questions how the same species can look dramatically different.
Answer: How can the same locust look so different?	Challenge about the Rocky Mountain locust's variable appearance. Asks how biologists could mistake different forms of the same species for different species, and seeks examples of highly polymorphic organisms.
SearchResearch (10/24/25): The shifting of Search...	Answer explaining species synonymization and polyphenism. Discusses how biologists confused locust phases (solitary vs. gregarious) for different species. Covers examples like African elephants being split into two species and eels' leptocephalus larvae.
SearchResearch (10/29/25): The 1 trick you need to...	Reflects on 15 years of SearchResearch blogging and 6.23 million reads. Notes the shift from traditional search skills to AI-augmented research methods. Announces format changes to focus on sensemaking and deep research skills rather than weekly challenges.
SearchResearch (11/5/25): Pro tips on using AI for...	Pro tips for using AI to enhance reading comprehension. Shows how to use LLMs to explain obscure phrases in P.G. Wodehouse novels by providing book context. Warns about hallucinations even in seemingly accurate explanations.
SearchResearch (11/13/25): How good is AI at recog...	Challenge asking how good AI is at recognizing images. Presents a dessert display photo and asks readers to identify the type of desserts and cafe origin using various AI image search tools.
Answer: How good is AI at recognizing images? What...	Answer post about AI image recognition capabilities. Tests Google Image Search, Bing, and Gemini on identifying a Middle Eastern dessert case photo. Reveals inconsistent results and hallucinations, emphasizing the need to verify AI visual analysis claims.
SearchResearch Method: Control-F for reality (find...	Introduces using AI (Gemini) to scan bookshelf photos for book titles. AI successfully locates specific books and provides directions to their shelf location. Can also analyze reading habits and suggest related books based on your collection.
SearchResearch (12/17/25): Control-F for reality--...	Tests 'Control-F for reality' using AI to find decaf tea on grocery shelves. Compares Gemini, ChatGPT, Perplexity, and Claude results, finding significant variation (3-14 decaf teas identified). Demonstrates that asking for spreadsheet output yields better results than simple lists.
SearchResearch (12/19/25): An experiment in cross-...	An experiment cross-posting between SearchResearch and the Unanticipated Consequences Substack. Uses NotebookLM to generate a 14-minute podcast about 'Why is thinking about the future so hard?' Asks readers for feedback on AI-generated podcast quality.
SearchResearch (12/24/25): Living in an AI world t...	Explores AI OCR accuracy for book spine recognition. Tests Gemini and ChatGPT on reading bookshelves, finding ~90% accuracy with hallucinations and missed titles. Offers tips: use high-res images, request 'Can't read' markers, ask for spreadsheet format, and always verify AI results.

Or, for a more global overview, I asked Claude for 250 word summary, which gave me this--which I think beautifully captures the overall sense and direction of SearchResearch in 2025.  

SearchResearch 2025: A Year of AI-Augmented Research

The SearchResearch blog in 2025 documented a pivotal transition from traditional search skills to AI-augmented research methods. After 15 years and 5 million views, author Dan Russell acknowledged that while foundational skills like Control-F remain essential, AI tools are transforming how we investigate questions.

A dominant theme was rigorous testing of AI capabilities—and limitations. Multiple posts evaluated Gemini, ChatGPT, Claude, and Perplexity on tasks like image recognition, OCR accuracy, and summarization. The recurring finding: AI achieves roughly 90% accuracy but still hallucinates, requiring constant verification. The mantra "double-check everything" appeared throughout.

The blog introduced innovative methods, notably "Control-F for reality"—using AI to search physical spaces like bookshelves or grocery shelves via photos. A three-part series explored AI-powered just-in-time learning using chatbot conversations and NotebookLM for audio/video summaries.

Traditional SearchResearch Challenges continued but less frequently, covering diverse topics: locust polymorphism, plant mimicry, cyclorama art installations, greenhouse history, and Prussian coffee sniffers. These demonstrated that curiosity-driven research remains central, even as tools evolve.

Russell explicitly announced a format shift: fewer weekly challenges, more focus on sensemaking and deep research skills. The blog increasingly addressed when to use search versus AI, concluding they're complementary—search for current facts, AI for synthesis and explanation.

The year's trajectory reflects broader changes in information literacy: mastering AI tools while maintaining critical evaluation skills that have always defined effective research.

 

But I wasn't done with this.  I turned to my old friend NotebookLM and asked for 1-slide visual summary of the year in SearchResearch.  (See below.)  Nicely done, even if it used only 35 of the 55 blog posts... 

Or, if you prefer, here's the MindMap that NotebookLM created.  (Click to see full size.)  

SearchResearch Lessons

As I've been saying, you have to check your work.  Plausible looking output needs to be verified.  Do not just look at the first one or two outputs and hope that everything else makes sense.  Sometimes it won't. (And sometimes you'll get penalized for that mistake.  Link to story about lawyer getting a huge fine for using ChatGPT for references in a legal brief. Don't be this guy.)  

And, secondarily, try Claude for your data-transformation tasks.  I've found it on this (and other, similar tasks) to be much more reliable.  

But.. Keep searching!