Friday, March 11, 2016

Why research skills matter more than ever

I watched in dismay as the young student slowly typed her query into the smartphone.  This was not going to end well…

She was trying to find out which city was the capital of the Belgian Congo in 1923.  She very reasonably searched: 

    [capital Belgian Congo] 

and in less than a second she discovered that the capital of the Democratic Republic of Congo was Kinshasa, a port town on the Congo River.  That was a great query, and she happily copied the answer into her worksheet.  Done!  

Keep reading... 

But you’re probably as surprised as I was while watching her.  

The Democratic Republic of Congo is a completely separate country from the Belgian Congo, and Boma WAS the capital city of the Congo Free State and Belgian Congo from 1886 to 1926.  In that year, the capital was moved to Léopoldville (which was later renamed Kinshasa during the reign of Mobutu in 1966).  Knowing which city was the capital during which time period is complicated in the Congo, so I’m not terribly surprised by the wrong result. 

The deep problem here is that she blindly accepted the answer offered by the first result shown in the search engine as correct.  She didn’t realize that there’s a deeper story here.  

She didn't know that capitals can (and in central Africa, often DO) change frequently.  

We know this is what many students do—they do the first query that pops into their heads and run with it.  Double checking and going-deeper are skills that comes only with a great deal of practice… and perhaps only by having a bunch of answers marked as wrong on important exams.  

Key point:  Students often don’t have a great deal of background knowledge to flag a result as potentially incorrect, so they’re especially susceptible to misguided search results like this.  She hadn't yet learned about the political turmoils of Africa, so she never thought to read more deeply.  

If she had clicked through to the landing page, she probably would have started reading about the history of the Belgian Congo, and realized that it has a few hundred years of wars, corruption, changes in rulers, shifts in governance.  The name of the country changed at least 6 times in a century, but she never realized that because she only read the answer as presented in the first search engine result. 

Teachers see this kind of thing all the time—students will sometimes write learned essays that reflect holocaust-denial versions of the holocaust because the top three hits from their Google query happen to be all revisionist versions of World War II history… and they don’t know they should go a little farther in the results list.  

You can try it: the query [crematorium holocaust review] gives revisionist results in the top 3 positions. If you’re a student, which results do you use?  If you're just now learning about World War II, how do you know these are bogus results? 

This is one of the reasons I teach people how to do research—there’s a lot of work to do here.  And it’s more than just students in school who make basic mistakes.  City councils have banned the use of water because they did poor basic research.  

The bigger problem is that relatively few people go deep enough to check on their basic research, and so we get ridiculous Facebook posts that propagate stupidities of the past that a minute’s worth of checking would reveal as silly.  (Just check for the latest depth of stupid for examples; they do excellent work in debunking the latest round of ridiculosity circulating on the net.)

Basically, when doing research, people keep making mistakes—and there’s no real place for them to learn how to do it correctly.  Very few schools are doing this—there’s just little capacity for teaching these skills, even though they’re critical. 

This is the reason I teach people how to do research—there’s a lot of work to do here.  

It’s obvious that very few students (or adults!) ever get to have a class about how to do research.  With search engines, we can make somewhat convincing research mistakes rapidly.  Speed is important, but with rapidity also comes a kind of carelessness that people often don’t recognize.  (Typical speed-induced errors:  not double checking; looking for confirmation, not contradiction; not knowing what kind of resources are available; not understanding the scope and credibility of a resource.)

Search Lessons 

1.  You have to keep your skills fresh as search changes.  Search engines are constantly creating new capabilities such as Search-By-Image.  Stay on top of these things! 

2. Realize that the underlying content changes.  What we believe to be true is constantly shifting (Example: How many planets are there?  8?  9?)   
3. Collections constantly change. The resources available to you changes constantly as well.  The number of books in full-view constantly changes, new image collections come online, new cultural resources spring into existence... 

4. New KINDS of stuff is out there, pay attention. Now you can easily find data sets, satellite images, Vines, podcast series, etc. What's the next newest content type you can search for?  

5. The dark side: new scams and bogus content are constantly arising as well.  You need to learn how to recognize this.  There seems to be a new conspiracy theory of the day, along with scam online pharmacies that can kill you with improperly made drugs, and disinformation sites.  

As I said, this is really why I write the SearchResearch blog, why I teach classes, and why I try to demonstrate how good research really works.  It's deeply non-obvious, and we all need to really understand how this stuff works.  


  1. Great stuff, Dan! I am just finishing a unit with my 8th grade computer class on web site evaluation and the IHR site is one of the ones I give them to examine. About 98% of the time they can't tell it is a Holocaust denial site, which leads us into a great discussion about the limitations of 8th grade life experience stacked up against the Internet.

  2. a curious, motivated mind will find a way… even without a search engine - just ask Larry & Sergey…
    but a lazy, mis-directed, unwilling, non-interested mind is a tall hill…
    it would seem to be a Sisyphean task, but instead of hill, it (a small iceberg instead of a rock) floats on quicksand that is perpetually shifting around small nodes of solid ground
    "It's deeply non-obvious, and we all need to really understand how this stuff works. " true dat
    calls to mind sayings like: (non-bing/googleable…)
    "you can lead a horse to water, but you can't make them win the Derby… or something like that"
    "the best tools on a laggard's bench can't build the palace an inspired poor man's hands can"
    perhaps too many settle for being "sane"…
    “The love of knowledge is a kind of madness.”
    ― C.S. Lewis, Out of the Silent Planet
    on the other hand, the real/hypothetical missed-Boma example you cited might very well dazzle you with her social media adeptness and finesse and
    might even show dark web navigational skills that would take even you hours to suss out …? such is interest, motivation and peer interaction…
    some mulling material:
    tHEaTLANTIC 2008
    there's a DEbate for that
    this year
    The Shallows
    as a counterpoint notion for the sake of balance and perhaps some levity… but not to harsh anyone's mellow…
    for the purpose of reader decorum, I shaved the first 35± seconds… don't let the messenger distract from the message… all comedy
    is built on the foundation of truth.
    Tater — it's not TED

    keep teaching, preaching & searching on Dan… who left this iceberg here? I'll search that out… later…

  3. "I watched in dismay as the young student slowly typed her query into the smartphone. This was not going to end well…"

    Dr.D… it was a catchy opening line so I goooooogled it… you may have a promising alternate vocation in Manga/Anime…
    opening line: "I watched in dismay…"

    "It was a dark and stormy night" yours may not be Paul Clifford level yet,
    but it is still a pretty good hook:

    Sadao Maou SERP
    Sadao Maou 真奥 貞夫 Maō Sadao?) / Satan Jacob (サタン・ジャコブ Satan Jakobu? — just working @ MgRonald's
    … perhaps a mashup?: "I watched in dismay as the rain fell in torrents and puddled at the feet of the Demon King…" (that's what editors are for…)
    the consummate author at work: "It was a dark and stormy night; the rain fell in torrents — except at occasional intervals, when it was checked by a violent gust of wind which swept up the streets (for it is in London that our scene lies), rattling along the housetops, and fiercely agitating the scanty flame of the lamps that struggled against the darkness."

    1. Good to know I have a backup career, in case this whole Google thing doesn't work out...

  4. Dan - great post, which I've just tweeted - we can only hope to chip away at this problem.

    I have just finished an online course on FutureLearn called "MAKING SENSE OF HEALTH EVIDENCE" by Cardiff University. It was very good and I can recommend it for when they run it again if anyone is interested in understanding and approaching critically health information.

  5. …meanwhile, in Seoul, we watched in dismay and awe… This was not going to end well… or maybe it would be OK… or maybe it would be stupendous…
    or maybe it would be of no real lasting consequence at all… or maybe the end had occurred and all that was left was for the curtain to fall…
    the mystery was that we could not discern the next move, but somewhere, out of sight, a circuit opened and a small black stone rolled off a table. cheers, Sergey

    3 of 5
    4 of 5 - human, 3-1 AlphaGo (to make it sporting)
    see 5 of 5, 3/15

  6. Dr. Russell,

    As a high school librarian, this is the #1 thing I try to teach students; that you simply can't do a search, look at the first 1-3 hits, and call it a day. For starters, the info could be wrong, as you've shown. Secondly, you have no context for the answer so the likelihood of remembering/retaining is low. Without an actual path/search, the answer is simply an answer. There is no deeper meaning, so way to contextualize it. It is A to Z instantly instead of A to Q to G to X and finally to Z. Not only do you have a context for the answer but one has also (ideally) learned something else along the way.

    All of this you know, I'm sure. There is this great bit by Pete Holmes ( that I encourage you to watch. I want to show this clip to every student in the world.

    The question, for me, is how to be encourage/foster curiosity? Can it be taught? How do we convince students to slow down, enjoy the process, and come away with a deeper understanding?

    I'm rambling, I know, but this is stuff I think of and deal with on a daily basis.

  7. I'm reading this a bit late, but feel compelled to comment. Not only should we bemoan the weak research skills demonstrated by this student, we should also despair about the quality of the question. Why should a student 'know' the capital of the Belgian Congo in 1923 as a discrete chunk of information? Why isn't the teacher asking the student to explore the consequences of European imperialism in the Belgian Congo? Educators set themselves up for lazy research and poor results when they ask only easy questions.