Why knowing search isn't the same as having an education

I don’t often get ideas from the comics page, but the Doonesbury strip of June 26, 2011 made me think about the fundamental goals of education and how people think about these things.   (Link to original image.)  

In the cartoon, Zipper is asked a series of trivia questions—“When is Guy Fawkes day?”  “What is the seventh most abundant element in the earth’s crust?” and “Name the three main branches of moral philosophy.”  Using his favorite search engine, he’s able to answer any question in milliseconds, even the one fairly deep question about moral philosophy. 

Ah… wait a minute… that one about moral philosophy is a pretty trivial question too; it doesn’t really need much inference or thought, just a quick lookup to find that the answer is “meta-ethics, normative ethics and applied ethics.” 

But wait another minute… it turns out there are actually five branches of moral philosophy—typical descriptions of moral philosophy include “moral psychology” and “descriptive ethics” as the fourth and fifth branches.  Other writers include other areas of ethics and there are debates around what is the true nature of moral philosophy abound because that is, after all, what philosophers do.  

The cartoon is funny because it satirizes the idea that education can be reduced to looking up facts and spitting them back out at blinding speed.  What I’m worried about is that not everyone sees this as a joke. 

Most people would agree, I hope, that factoid lookup is not *really* what education is all about.  And yet that is in many ways the box into which our debate about education is being forcefully stuffed.  We test educational achievement and evaluate the progress of schools by giving students standardized tests that ask questions like “name the three main branches of moral philosophy.” 

As a real example, here are a couple of representative questions from the California Standards test for 8^th graders: 

   1.  Tasha is buying a CD that is regularly $12.99 and is now for sale at 25% off. 

        Which expression can she use to estimate the discount on the CD? 

     (A) 12.99*0.25  

     (B)  12.99/25 

     (C) 12.99+12.99*0.25

     (D) 12.99 – 12.99/25    

   2.  The Declaration of Independence elaborates on the Enlightenment idea of:

     (A) natural rights

     (B) collective ownership

     (C) religious freedom

     (D) political equality

These tests reinforce the notion that education is the sum total of what you’ve acquired during your time in school.  Students naturally ask “what’s the point when I can just look up these things?”  They have a point.  If you construe learning and time in school as the gradual accumulation of factoids that you have to recognize on a test, then it’s a fair pushback—why deny them the tools they’d use in real life?   That is, assuming real life / real education is looking up factoids about the world. 

But the tests aren’t really testing knowledge, they’re an instrument that measures the marks of education on the student.  They’re measuring the side-effects of learning.  If you’ve really learned much about the American Revolution, then you probably know what the Declaration is all about.  This question is a proxy metric: it doesn’t measure how much you’ve learned, but is an estimator that samples the shape of scholastic tire tread left on your brain by the educational school bus.   If you’re actually trying to understand how the Declaration of Independence informs and influences modern political thought, you need to know what the Enlightenment was in a broad sense and how it shaped the Declaration; you probably don’t need to know which Enlightenment idea is elaborated by that document.   (Or worse, what you think the test writers thought was the idea.)  But that’s harder to test at the state-wide scale we need. 

Rapid lookup is a great tool, but it’s NOT education or learning in any meaningful way.  The framework that organizes all those factoids and inter-relationships IS education—it places all the bits and pieces into context and lets you understand the structure and functions of the world. 

My first reaction to the Doonesbury comic was to think “How did he know there ARE three major branches of moral philosophy?”  That is, how did he know enough to frame the question in the first place?   What was the framework the questioner had that allowed him to even think of the question? 

It’s true that more-and-more content is available online, and that students should make use this wealth of resources, that isn’t for up debate—of course they should.  Unfortunately, one ways to save money in tight school budgets is to remove school librarians as has been proposed in several states (e.g, NY and CA).  They don’t, after all, do much to improve test scores on factoid tests and don’t have a natural constituency (unlike, say, 4^th graders teachers, which DO). 

Instead, I would argue the opposite: the rise of increasing amounts of online information should INCREASE our teaching of information literacy, not decrease it.  Sure, many K12 teachers teach a bit of how to work in an online world, but the education is spotty and in many cases not especially good. 

I dislike the term “digital literacy” that’s sometimes used to describe this because the “digital” part is just a descriptor of how the information is captured… it describes the means and not the strategies and tactics.  It’s no more “digital literacy” than to call its precursor “paper and ink literacy.” 

I prefer (but am not 100% committed to) “information literacy” as the term for the skills a student needs to develop.  Some information is digital, a good bit is not.  Do I suddenly stop dealing with my problem because what I seek is no longer in a digital medium?  Of course not.  The idea of “information literacy” is that these skills transcend any particular medium.  In fact, I’ll go farther and say the strategic skills are true across many different aspects of information.  My favorite skill:  “Know when to stop searching and find a human expert on the topic.”  You’d be surprised how often people don’t know when to employ this strategy!

In general, someone needs to teach students that just looking up X isn’t the same as understanding the context of X—where does X come from, why is it important, what super-categories of X exist, does X vary from culture to culture?  (…and so on – you can imagine asking these questions about our “moral philosophy” example from above: “Where does moral philosophy come from?  Does moral philosophy vary from culture-to-culture?”) 

And someone needs to teach students that just looking up Y doesn’t tell you how to assess and evaluate that information—the text of Y comes from somewhere and someone… who wrote that about Y, why did they write Y, and how does what you read about Y fit into what you already know? 

You’ve heard this before, but it’s increasingly true:  Much of being a sophisticated learner is not just knowing how to look up something, but being able to ask the right questions at the right time to advance your understanding.  That’s the key to good research.  Good questions stem from a deeper understanding of the domain, and not from lightweight and cursory perspective.  And most people don’t know how to ask good questions based on what they’ve read and seen in online materials. 

I mean that seriously:  You’d be surprised.  Really.  It’s something we need to fix as a culture. 

Answer - What's happening with employment in California?

When I asked "What's happening with employment in California?" I was purposefully framing the question in a way that someone might think of it.  That is--I did NOT prime you with the search term [ unemployment ].  


If, however, you managed to see your way through that obstacle, you might have done a query like this:  


[ California unemployment rate ] 


Which would bring up a Google shortcut result in position one, like this:  

This is the "Public data" result, which leads you into a marvelous compilation of public data that's been collected and organized by the same Google team that's brought you the bubble charts in Google Spreadsheets.  


From our perspective, though, it's a single-stop for comparing unemployment rates by county within California (or any other state).  In particular, we can simply answer the question I asked earlier (that is, compare the employment rates between Santa Clara and Santa Cruz counties).  


With three mouse clicks, you can select the data from Santa Clara and Santa Cruz (and deselect all of California) to produce this graph: 

(Teachers:  A good question to ask is "Why is Santa Cruz county so much more variable than Santa Clara county?"  The answer has to do with what each county does economically--Santa Cruz is much more agricultural, hence the seasonal swings as ag-workers are employed or unemployed.)  


Note that you can hover over a particular data point to drill-down to specific dates. 

In this particular case, the data is supplied by the US Bureau of Labor Statistics (which you can discover by clicking on the link in the lower left of the graph).  As always, you want to verify that the data comes from a reputable source.  


Is this just a trick?  I've had some people complain to me in the past that queries like this are "just tricks... there's nothing methodical about it, you just have to know a lot of stuff." 


And, to a certain extent, that's true.  But this is the nature of expertise--to be a real expert at something takes practice and knowledge.  The wonderful thing about reference librarians is that they already know a great deal about how the information in the world is organized.  They know what resources exist and what's possible to do with them.  


My deeper point in this post is to recognize that sometimes you can stumble across resources (such as the Public Data shortcut I've shown you here).  When this happens, take note. Knowing where the data is kept and how to access it really is a fairly important part of becoming a great researcher.  
_________

You can learn more about the range of shortcut results at the Google Helpcenter page Explore Google Search.

And, to learn more about the other kinds of public data that's organized by Google, see the Public Data Explorer site, which will connect you to data about World Economic Development, birth rates, unemployment (US and EU), cancer causes, and many other wonderful data sets.

Special note for teachers:  These datasets are great for teaching concepts about statistical analysis.  

Chaining searches / how kids search / teaching kids how to search

Yesterday's search challenge wasn't much of a challenge. The simplest search solution I know is to first find the Gladwell article on basketball with a query like:

[ Gladwell basketball ]

The first result will be the New Yorker article.  Clicking then on that article you'll quickly find that the CEO we're looking for is Vivek Ranadivé.  The next search should be obvious:

[ Vivek Ranadivé  ]   

And you will quickly see that Vivek Ranadivé is the CEO of Tibco.

As I said, this isn't that hard of a challenge.  All you had to do was to:

1.  Find the Gladwell article and skim it, looking for the coach's name.
2.  Search for the coach's name, then look for the company name. 

The reason I wanted to start off with a simple problem like this is that this "chaining searches" together to solve a problem is a fundamental skill--it's something you actually have to learn at some point in your life. 

When I go out to teach at elementary and middle schools, I'll often find that kids will have a hard time creating their own chain of reasoning like this, at least until they're in the 6th or 7th grades.  It doesn't seem to be a matter of intelligence as much as practice in working out the search chain. 

In her work at the Univeristy of Maryland, Allison Druin (and colleagues) often use "The Vice-President's Birthday" problem to assess this chaining skill. 

The problem is this: 

"On what day of the week (Sunday, Monday, Tuesday...) will the Vice-President's birthday be next year?"

Again, it's not that hard: 

1.  What's the vice-president's name?
2.  Once you have the name, lookup his birthday (day-month-year). 
3.  Once you have the date, search for a calendar for next year (that is, this year + 1).
4.  Find the date, read off the day-of-the-week. 

But figuring out that this is the sequence of steps you must go through to figure out the weekday is just a bit over the heads of many 7-to-11 year olds.   As Druin (et al.) say "Despite experience with searching, children tended to fall back on... natural language queries..."  and points out that  "Frequently those natural language queries were the verbatim questions asked by the researcher." 

To the eye of an expert searcher, this sounds crazy--but we frequently see people in our studies searching for whole phrases that repeat much of what you have asked them to find. 

However, the skills of seeing to the essential core of a question and divining search terms is a real skill, and there is a fair bit of evidence that adults basically do much the same thing. 

So, teaching a kid how to search effectively is partly one of showing how to choose search terms (and almost nothing about Boolean ANDs or ORs).  and partly one of how to break up a complex problem into a sequence of easily achievable substeps.  Those two skills interlock, as you can see in my favorite kid query for this problem:  [ vice-president birthday Sunday ]   He then went on to repeat this query with Monday, Tuesday, Wednesday... He was good-natured about it and gave the wry explanation that "well... if his birthday IS on a Sunday, then I'll find it." 

Maybe.  But it's not a good strategy to follow.  A much better way is to devise a search that will yield an answer that can be used in the next step, and not to test all of the available options.  (Imagine if I'd asked "when was the last year of the Civil War?"!) 

More on how to good search strategies in times to come. 

--
Allison Druin (et al.) paper on how kids search is available at her site at University of Maryland:  "How Children Search the Internet with Keyword Interfaces."   (This will be published at the CHI 2010 conference later this year.)