What do you believe about the data?
In the previous post I wrote about mental models of systems—that is, how you believe an information retrieval tool or web search system operates. Is it okay that you think it's all hamsters all the way down?
But the other side of the coin is your mental model of what data the system is operating with. Tool operation is important, but tools operate on something, and understanding the properties of something is just as important.
Let me ask you, “what is your mental model of the information that search tools operate on?”
Think of it along 3 dimensions:
1. Completeness: How much of the total information space out in the world is available to you by search? Is it complete?
2. Kinds: What different KINDS of information can be found?
3. Origin: Where does the information come from?
In my research I’ve found that searchers usually have a relatively limited mental model of the information space. As you can extrapolate from the previous post, people think of Google as working on either a GIANT library of content, or a GIANT database with structured content in it.
The truth is, neither is true. Google indexes all of the web pages it can reach by crawling (following links from one page to another) OR data that it gets from specialized feeds (like news from publishers). Google Scholar is a great example of both—they crawl the web looking for texts that look like scientific papers, and they get data dumps from specific publishers. There are also content sites that Google maintains like YouTube for videos or Google Play for books, apps, and games.
 |
| The entire corpus of human knowledge is NOT what Google is about. It's a sprawling, complicated mess. P/C by Gemini. |
As a consequence, people believe that everything—all of the knowledge of humanity—is indexed and available with a simple search. During my studies I’ve had multiple people tell me that “everything is in Google.” (You can read more about this in the paper In Google We Trust: User’s decisions on rank, position, and relevance.)
This research result also tells us that users exhibit a strong "trust bias" toward search engine rankings. Participants consistently assumed that if a page was at the top of Google, it was the definitive answer. Users frequently assume that if something cannot be found on Google, it does not exist.
For more insight on this, see: Transparent queries: investigate users' mental models of search engines
We know that searchers operate with simplified mental models. They tend to view search engines not as specific, curated index databases with technical limits, but as direct windows onto the entire universe of human knowledge.
Unfortunately, that’s not even close to being true. People assume Google crawls 100% of the live web in real time. In reality, Google’s index represents a tiny fraction of the total web content (often estimated at less than 5%–10% when accounting for the Deep Web).
What’s more, in actual use, searchers tend to use the tools they have in fairly shallow ways. We’ve known for a while that the vast majority of users never look beyond the first page of results (often called the "principle of least effort"). Users unconsciously equate unindexed or low-ranked with non-existent. (See: "Search engine use behavior of students and faculty”)
As a consequence, searchers often believe that If a query yields no results, they tend to infer that the object, fact, or paper does not exist anywhere, rather than recognizing that Google's crawlers simply haven't discovered, indexed, or rendered it.
People also frequently expect Google to retain historical versions of every page forever, confusing web indexing with digital archiving (like the Internet Archive’s Wayback Machine).
What Google Can Find
1. Crawlable Surface Web Content: HTML pages, publicly accessible blogs, news, public forums, and open-access documents.
2. Specific Indexed Document Types: PDF, DOCX, PPT, XLSX, TXT, EPUB, and specialized formats. (But also note that the site: operator will find a file with that extension no matter what the extension is. Example: [filetype:randomness] – this isn’t a real file type extension, but Google will find documents with this extension!)
3. Structured Data & Knowledge Graph Entities: Information formatted via Schema.org (recipes, events, products, flight statuses, explicit facts about known entities). Often imported directly rather than crawled.
4. Media Metadata: Images (via ALT text, surroundings, and computer vision), videos (via transcription and timestamps), and audio metadata.
5. Renderable Dynamic Content: such as JavaScript-rendered pages (Client-side rendering, React/Vue applications) that the Googlebot can execute.
What Google Cannot (or Does Not) Find
A. The Deep Web (walled-garden Content): Information behind paywalls, subscription databases (e.g., academic databases like JSTOR, LexisNexis), private intranets, or user login screens.
B. Dynamic Database Content: Data generated dynamically in response to form submissions (e.g., searching a specific library catalog, internal flight databases, or government record systems directly).
C. Unlinked / Orphaned Pages: Pages without inbound links or explicit sitemaps that web crawlers have no path to discover.
D. Blocked Pages: Pages explicitly marked with noindex or blocked from the Googlebot crawler via server directives.
E. Ephemeral & Real-Time Data: Real-time messaging (Slack, Discord, private WhatsApp threads, live social streams) that isn't publicly archived.
F. The Dark Web: All of those non-standard networks requiring specific routing protocols (e.g., the .onion domains). There are other ways to find this information that you can figure out.
SearchResearch Lesson:
What all of this means is that user mental models for online search (or for any AI system!) need to include what’s being operated on IN ADDITION to just a model of how the system functions.
To be an effective and savvy searcher, you need to know not just about how the tools operate, but also the nature of the content being indexed.
You need to know, for instance, that you can search for an image by using Google Lens, but that Google Lens will not find your image in a single frame of a video (even though the video is composed of thousands of image frames arranged in a long linear list)!
But you knew that, didn't you?
Keep updating your mental model and...
Keep searching.
No comments:
Post a Comment