We must go deeper: Why search is always going to be tricky

In the movie “Inception,” the main character keeps goading his team farther into the subconscious mind with “we must go deeper…”  Their goal is to get to the bottom of the many layers of experience that makes up the human mind in order to plant a single idea. Each layer depends on the ones below it; they pile up to make an image of perceived reality.  And that’s kind of what I’m thinking here…

I often have this feeling of “we must go deeper” when I’m doing a simple search for information on Google—and yet I’m surprised when I see that other people don’t seem to have that same urge to understand what’s really going on, but are satisfied with the first answer that pops to the top. 

Somewhere along the way I must have developed a fact-checker’s outlook.  You know who the fact-checkers are: they’re the woefully underpaid people who read news, books, and magazine articles pre-press and meticulously verify that what’s written actually lines up with observable reality.  They’re paid to not just lookup the original publication of a fact, but then to also follow-up with the next few issues to see if any retraction or correction was published.  That’s what a real fact-checker does.  Running down truth, even if it takes several extra steps.

The simple reality is that looking stuff up has always been difficult.  Now, it’s just difficult in a different way (so many alternatives to consider, so many ways to think about things, so many ways to measure).  A fact-checker looks at a sentence like “There were 31 bars in Elko, Nevada in 1965” and immediately asks the question “How would I figure this out?  Would I count liquor licenses issued in the city of Elko during ALL of 1965, or just active licenses on 12/31/65?” 

When I talk to students as they search for simple facts, I find that they just want to get an answer that’s plausibly correct.  The actual truth of a fact doesn’t seem to matter as much as it confirms something previously known, or is merely close to something plausible.  It’s as though they live in an ongoing game of horseshoes where getting your token close to the post is often good enough to win.  

The thing that worries me these days is that not only do students not want to second source a reference (let alone look for retractions or corrections), but they often don’t want to understand the topic in enough detail to know how-and-why their fact might be incorrect.

The simplest example I can think of is the straight-forward question “What’s the circumference of the earth?”  How hard could finding that be?

 It IS the case that if you do a quick Google search, you can quickly end up with a very credible looking answer.  Here’s one answer: 24,901.55 measured around the equator (red arrow).  But here’s another:  24,859.82 miles if you measure around the poles (yellow arrow).  And of course, you can get different sets of number depending on who made the measurement, how it was made, and according to what model of the earth they’re using.

Now wait a second!  How does “model of the earth” enter into this discussion?  Isn’t the circumference an actual physical measurement? 

Well, yes, it is… but this is where you have to go deeper and realize that nobody actually takes the measurement.  Think about it for a second:  Walking around the equator with a yardstick just doesn’t work for all kinds of practical reasons.  So to actually measure the circumference, you’d actually measure something that’s easier to do—say, the difference between two vertical angles measured at two points on the surface.  (That is, if you project a vertical ray up from a point and see what star it hits, then walk north 100 miles and project a second vertical ray upwards, you can measure the difference in the angles and figure out what angle is subtended.  Once you know the diameter from other means, with this measured angle you can work out the circumference of the earth.) 

But here’s the tricky part:  If you measure some fraction of the earth’s surface, you then drop that into a MODEL of the earth to compute the circumference.   If you assume the Earth is a sphere, then it’s just simple geometry.  But  the truth is, you have to know that the earth isn’t a sphere.  (I assume you know that!)  But if it’s not a sphere, then what IS it?  That’s where the model comes in. 

Newton thought the earth was an oblate spheroid (a sphere that’s been squashed down a bit) with a flattening of 1:230 from truly spherical.  He derived that ratio from his model of gravity, and brilliant though it was to figure that out, it’s not quite right.  Measurements later showed that the flattening is more like 1:210.

Except… we must go deeper…

The model of Earth as oblate spheroid assumes that it’s a “body of revolution.”  That doesn’t mean it suffers from constant political uprisings, but means the model assumes it’s pretty smooth, as though it were turned on a lathe.  Of course, you know by now that nothing is that simple.  Earth is actually fairly lumpy, with bumps on it—think Rocky Mountains or Himalayas—or low spots like Death Valley.  So a better model is complex ellipsoid with various undulations that make it NOT a body of revolution. 

The real question is probably this: Why should I care?  The two numbers given above (24,901.55/ 24,859.82) are within 1% of each other.  Does that matter? 

If you’re launching a spacecraft, it matters.  But the real point is that when searching for something as straightforward as the “earth circumference,” there’s a backstory about the number that reveals a good deal.   (And personally, the backstory is often more interesting than the final number… but I digress.) 

Thing is, this is true for most things you look up.  Suppose someone asks “what’s the biggest city on Earth?”  A quick Google check shows that it’s Shanghai with 17,836,133 people.  Really?  Since the time you started reading this, some people died in Shanghai while others were born.  What does the instantaneous population of a city even mean? 

And so checking the backstory quickly leads to clarifying questions: “What do you mean by biggest?”  Are you measuring population or area?  Do you mean city boundary as determined by the city council or by measuring the urban core and excluding suburban regions? 

This is, of course, what reference librarians do—they conduct the “reference interview” to try and pin down the variable parts of someone’s question. 

When I was doing a field study at the University of Alaska (Anchorage),  I actually played the part of reference librarian for a couple of hours while the snow fell and it grew dark at 3PM.  Mostly the questions were “where’s the bathroom” (no backstory needed for that!) and “how do I get my printer to work?” 

But then a graduate student asked for a “list of the top ten journals in Sociology.”  He wanted to get a general awareness of what the field was about and what would be reasonable topics of current interest. 

Getting a list of sociology journals isn’t hard.  Rank ordering them according to some reasonable function is tough.  “What would you like?”  I asked.  “Readership size?  Citation rates?  Agreed-upon influence?  Most-often-checked-out?”  Those all produce somewhat different sort orders.

Thing was, he didn’t really care.  He didn’t need an ordered list—just ANY list of 10 good journals would be fine, thanks very much.

I’d just assumed that he wanted an ordered list.  That’s what working at Google does to you; I see the world as a ranking problem and just assumed he wanted a list with an evaluation.  But since he was going to spend as much time with journal #1 as with journal #10, it really didn’t matter. 

Any good psychologist will tell you that “the presenting problem is often not the root problem.”  Likewise, the first question is often not the real question.  Good searchers take a problem and worry on it a little.  Good searchers look behind the “answer” to understand how that answer came to be.  In the circumference case, it was the measurement method.  In the population problem, it was how “largest” was defined and then how that property was measured as well. 

If I could somehow give everyone a piece of search advice, I’d say “think like a fact-checker” and get one or two steps deeper into your topic matter.  You must go deeper in order to understand what it is you think you’ve found.  The fact you seek isn’t just a fact, but something based on a set of choices about how to measure and what to report. 

Of course, I can say this all I want.  But this approach hasn’t worked well in the past.  (People have been saying this for a long, long time.)  How can I help people see this is what they really want to do when they really want to understand a topic? 

Ideas?