Monday, August 1, 2011

Why knowing search isn't the same as having an education


I don’t often get ideas from the comics page, but the Doonesbury strip of June 26, 2011 made me think about the fundamental goals of education and how people think about these things.   (Link to original image.)  


In the cartoon, Zipper is asked a series of trivia questions—“When is Guy Fawkes day?”  “What is the seventh most abundant element in the earth’s crust?” and “Name the three main branches of moral philosophy.”  Using his favorite search engine, he’s able to answer any question in milliseconds, even the one fairly deep question about moral philosophy. 

Ah… wait a minute… that one about moral philosophy is a pretty trivial question too; it doesn’t really need much inference or thought, just a quick lookup to find that the answer is “meta-ethics, normative ethics and applied ethics.” 

But wait another minute… it turns out there are actually five branches of moral philosophy—typical descriptions of moral philosophy include “moral psychology” and “descriptive ethics” as the fourth and fifth branches.  Other writers include other areas of ethics and there are debates around what is the true nature of moral philosophy abound because that is, after all, what philosophers do.  

The cartoon is funny because it satirizes the idea that education can be reduced to looking up facts and spitting them back out at blinding speed.  What I’m worried about is that not everyone sees this as a joke. 

Most people would agree, I hope, that factoid lookup is not *really* what education is all about.  And yet that is in many ways the box into which our debate about education is being forcefully stuffed.  We test educational achievement and evaluate the progress of schools by giving students standardized tests that ask questions like “name the three main branches of moral philosophy.” 

As a real example, here are a couple of representative questions from the California Standards test for 8th graders: 

   1.  Tasha is buying a CD that is regularly $12.99 and is now for sale at 25% off. 
        Which expression can she use to estimate the discount on the CD? 
     (A) 12.99*0.25  
     (B)  12.99/25 
     (C) 12.99+12.99*0.25
     (D) 12.99 – 12.99/25    

   2.  The Declaration of Independence elaborates on the Enlightenment idea of:
     (A) natural rights
     (B) collective ownership
     (C) religious freedom
     (D) political equality

These tests reinforce the notion that education is the sum total of what you’ve acquired during your time in school.  Students naturally ask “what’s the point when I can just look up these things?”  They have a point.  If you construe learning and time in school as the gradual accumulation of factoids that you have to recognize on a test, then it’s a fair pushback—why deny them the tools they’d use in real life?   That is, assuming real life / real education is looking up factoids about the world. 

But the tests aren’t really testing knowledge, they’re an instrument that measures the marks of education on the student.  They’re measuring the side-effects of learning.  If you’ve really learned much about the American Revolution, then you probably know what the Declaration is all about.  This question is a proxy metric: it doesn’t measure how much you’ve learned, but is an estimator that samples the shape of scholastic tire tread left on your brain by the educational school bus.   If you’re actually trying to understand how the Declaration of Independence informs and influences modern political thought, you need to know what the Enlightenment was in a broad sense and how it shaped the Declaration; you probably don’t need to know which Enlightenment idea is elaborated by that document.   (Or worse, what you think the test writers thought was the idea.)  But that’s harder to test at the state-wide scale we need. 

Rapid lookup is a great tool, but it’s NOT education or learning in any meaningful way.  The framework that organizes all those factoids and inter-relationships IS education—it places all the bits and pieces into context and lets you understand the structure and functions of the world. 

My first reaction to the Doonesbury comic was to think “How did he know there ARE three major branches of moral philosophy?”  That is, how did he know enough to frame the question in the first place?   What was the framework the questioner had that allowed him to even think of the question? 

It’s true that more-and-more content is available online, and that students should make use this wealth of resources, that isn’t for up debate—of course they should.  Unfortunately, one ways to save money in tight school budgets is to remove school librarians as has been proposed in several states (e.g, NY and CA).  They don’t, after all, do much to improve test scores on factoid tests and don’t have a natural constituency (unlike, say, 4th graders teachers, which DO). 

Instead, I would argue the opposite: the rise of increasing amounts of online information should INCREASE our teaching of information literacy, not decrease it.  Sure, many K12 teachers teach a bit of how to work in an online world, but the education is spotty and in many cases not especially good. 

I dislike the term “digital literacy” that’s sometimes used to describe this because the “digital” part is just a descriptor of how the information is captured… it describes the means and not the strategies and tactics.  It’s no more “digital literacy” than to call its precursor “paper and ink literacy.” 

I prefer (but am not 100% committed to) “information literacy” as the term for the skills a student needs to develop.  Some information is digital, a good bit is not.  Do I suddenly stop dealing with my problem because what I seek is no longer in a digital medium?  Of course not.  The idea of “information literacy” is that these skills transcend any particular medium.  In fact, I’ll go farther and say the strategic skills are true across many different aspects of information.  My favorite skill:  “Know when to stop searching and find a human expert on the topic.”  You’d be surprised how often people don’t know when to employ this strategy!

In general, someone needs to teach students that just looking up X isn’t the same as understanding the context of X—where does X come from, why is it important, what super-categories of X exist, does X vary from culture to culture?  (…and so on – you can imagine asking these questions about our “moral philosophy” example from above: “Where does moral philosophy come from?  Does moral philosophy vary from culture-to-culture?”) 

And someone needs to teach students that just looking up Y doesn’t tell you how to assess and evaluate that information—the text of Y comes from somewhere and someone… who wrote that about Y, why did they write Y, and how does what you read about Y fit into what you already know? 

You’ve heard this before, but it’s increasingly true:  Much of being a sophisticated learner is not just knowing how to look up something, but being able to ask the right questions at the right time to advance your understanding.  That’s the key to good research.  Good questions stem from a deeper understanding of the domain, and not from lightweight and cursory perspective.  And most people don’t know how to ask good questions based on what they’ve read and seen in online materials. 

I mean that seriously:  You’d be surprised.  Really.  It’s something we need to fix as a culture. 



8 comments:

  1. I am reminded of the recent research by Betsy Sparrow and team that concluded that people remember where they saved knowledge factoids (example- folder names) rather than the factoids themselves. It's as if we've become adept at using computers to store knowledge for us, and we're better at remembering where information is stored than the information itself. I am not saying this is bad, only that we need to think with expanded scope as to what would really constitute education in a 'lookup world' without rejecting it outright (if I can look it up, why should i be expected to remember it?). The fault is with testing, not with teaching or learning processes.

    ReplyDelete
  2. I agree with most of your points, but the math question was not a factoid lookup question. I doubt that google would help much in answering it. It was a question about understanding what percentages mean, which can't be quickly answered with a factoid, but which is trivial if you understand the concept.

    ReplyDelete
  3. The reason education cannot take place via search engines is that search engines do not produce the chains of logic upon which all science (i.e. all rational discourse, not just physical and/or experimental science) and education rests. It is not until we possess the entire chain of reasoning leading to a conclusion that we think we know the answer to any scientific question as a fact, while search engines produce only, at best, the conclusion. Hence, "education" in its proper sense cannot be gotten out of search engines. The best the engine can do is direct you to a page containing the logical chain in question, which you can then read and follow from start to finish and finally possess knowledge (or the current theory, at any rate) about it.

    ReplyDelete
  4. This xkcd comic is on the same theme: http://xkcd.com/903/

    I'm just writing my philosophy Ma dissertation on the implications of 'extended mind' for our understanding of knowledge & educational assessment of that knowledge; I think what probably changes is a shift from propositional 'knowing-that' knowledge, to skill based 'knowing-how' and meta-cognitive strategies...which is probably a desirable shift. Next step; psychology/education PhD to look at the empirical element!

    It's also interesting that in Denmark internet access has been allowed in a number of exams for the last few years(sans communication) - utterly different conception of assessment & knowledge.

    ReplyDelete
  5. A couple of quick comments in reply...

    Mohan -- My point is that you can't look up something that you don't already know exists. Nor can you reason effectively when every lookup takes you a number of seconds. (I'll write more on this in a future post.) Don't get me wrong -- lookup is *great*! But it's not a substitute for knowing.

    Gasstationwithoutpumps -- Oh, I don't know about that. Have you tried the query: [ how to compute discount ] ? The first result is basically the answer.

    Rumpel - What you say is true.... now. One can imagine a future in which logical inference does take place. This is how Watson (the IBM question-answering system that plays Jeopardy) works. But I don't think this solves the education problem in any way. There's still a "how to teach" component that's critical.

    S -- I should have known there'd be an XKCD on this. I did NOT know that Denmark allows this. Really? Can you post us a link to something explaining it?

    ReplyDelete
  6. Hi Daniel, some bits below, there seems to be remarkably little in English which is a touch frustrating obviously! Actually hoping google will give me some research money to look at some of these things.

    There is a rather nice video of Steen Lassen (a Danish ed minister) talking (in English) about it here: http://vimeo.com/8889340

    And:
    CISCO. (2009). Case Study: Danish National Assessment System. CISCO Systems. Retrieved from http://www.cisco.com/web/strategy/docs/education/DanishNationalAssessmentSystem.pdf

    Cunnane, S. (2011, May 12). The Danish gambit: online access, even during exams. Times Higher Education. UK. Retrieved from http://www.timeshighereducation.co.uk/story.asp?sectioncode=26&storycode=416090&c=1

    Undervisningsministerie (Ministry of Education), & Afdelingen for Gymnasiale Uddannelser (Department of Secondary Education). (2010, September). Endelig rapport fra følgegruppen forforsøget med digitale prøver med adgang tilinternettet i udvalgte fag på stx og hhx (Final report of the Monitoring Group experiment with digital samples with access to Internet in selected subjects at stx and HHX). Undervisningsministerie (Ministry of Education). Retrieved from http://www.uvm.dk/~/media/Files/Udd/Gym/PDF10/101007_it_rapport_13%20september%202010.ashx

    ReplyDelete
  7. Hi Dan
    Just returned to this as I'm still working in the area :-) (just thinking about writing a blog on the 'does google make us stupid/smart' clichéd question). Bit of a link fest, but v. happy to have returned to this post as it's so relevant to my interests!

    I used another video from BBC on the Danish example in a talk recently,
    Video: http://news.bbc.co.uk/1/hi/education/8341589.stm
    Talk: http://www.slideshare.net/sjgknight/so-lar-epistemology-pedagogy-assessment-v2 SoLAR Storm talk: epistemology, pedagogy, assessment

    My scoop.it at www.scoop.it/edu-search and blog (http://people.kmi.open.ac.uk/knight/) are also relevant on search, why it's interesting from an educational (and epistemological) perspective, and what it can/can't tell us about.

    Cheers!
    S

    ReplyDelete
  8. "Factoid", you don't have to spell the word, just give a definition, like this:

    Factoid was coined by Norman Mailer in his 1973 biography of Marilyn Monroe. Mailer described a factoid as "facts which have no existence before appearing in a magazine or newspaper", and created the word by combining the word fact and the ending -oid to mean "similar but not the same".

    ReplyDelete