Answer: Finding reviews

First, let me admit something to you.  Sometimes, when I post search challenges, I don’t necessarily know the answer.  I like it that way—it makes the challenge a real challenge for me too!  

That’s why I’m posting my answer to Wednesday’s challenge today, 2 days late—this was harder than I thought! 

So.. here’s my answer to the challenge.

On finding book reviews…

I have good news and bad news.

First:  good news—there are lots of collections of book reviews, written by scholars and organized into coherent clusters by topic.  There’s one for the social sciences, there’s one for psychology, one for education, one for physics, etc etc. 

Next, the bad news.  Many of the great compendiums of book reviews that people use for scholarly purposes are locked-up behind paywalls where it costs a good deal of money to get to them.  That is, there are many such databases—NOT indexed by web search engines such as Google or Bing—that you need access to if you want to use them.  BUT, they all cost money—in many cases, it’s prohibitive for an individual to gain access.

And in some cases, the reviews are available only in print.  Really. 

In any case, IF you’re a university student, your institution probably pays for you to have access to databases like the “International Bibliography of Book Reviews” (a collection of scholarly reviews going way back), or “Book Review Digest” (a collection of reviews that’s assembled annually), or “Book Review Index” (reviews from 1965 to present, but is very comprehensive).  Scholarly reviews of books can be found in other subscription databases such as "America: History and Life," JSTOR, and "Project Muse."  And for medieval literature specifically, the database "Iter: Gateway to the Renaissance and Middle Ages" is particularly helpful. 

And, the best summary of online AND print resources for book reviews is at the Cornell library’s site. 

http://www.library.cornell.edu/olinuris/ref/bookreviews.html 

Caution, though—all of the links from this page send you to the Cornell login page.  (Ahh, the value of an Ivy League education!)

Now for more good news:  As it turns out, some public libraries also have access to some of these resources, so you can check them by logging in through your library.  (And note a really big secret—you typically DO NOT have to live in a city to get a library card.  It’s worth getting library cards from cities other than where you live in order to get access to their databases!) 

But assuming you don’t have a good library card, some online resources worth checking:

New York Times Book Reviews http://www.nytimes.com/pages/books/index.html

One of best online review sites for the humanities is H-Net Reviews in the Humanities and Social Sciences:

 http://www2.h-net.msu.edu/reviews/home.php

Washington Post Book Reviews: http://www.washingtonpost.com/wp-dyn/artsandliving/books/  (free registration required). 

Lastly, I would be remiss if I didn’t point out the book reviews available through Google Books. 

If you search for the book on Google Books, a set of reviews that can be found through web crawling are listed at the end of the entry.  (See below) 

If that’s true that Google auto-collects book reviews, why did I go to so much trouble to look up other ways to find book reviews? 

Answer:  Because the reviews listed in Google Books often come from fairly random sources.  If you look at the reviews for “World Lit Only…”  you’ll see that they ALL come from GoodReads.com, which is fine, except that the reviews are very short (100 – 300 words on average) and are typically either full of praise for the book or an excoriation.  Critical analysis isn’t what GoodReads.com is good at—it’s really a list of book recommendations by other readers, and not necessarily scholars in the field. 

Since I already have more book recommendations than I know what to do with, GoodReads reviews aren’t what I need. 

And THAT’s why we need all the other book review sources.  For completeness and scholarship. 

FINALLY.. .the answer to my question:  How credible is the book “World Lit Only…”? 

I actually read about 30 reviews of the book from “regular readers” who generally loved it.  (It IS an exciting read with lots of salacious details about the Middle Ages—everything from popes going to war, orgies and lots of titillating details.)  But medievalists generally find it an egregious overstatement and hyperbolic without many references.  And they’re both right.  It’s a great read, but take the more outrageous statements (“peasants were sometimes reduced to selling their clothes and working in the fields completely nude..”) with a huge grain of salt. 

But (bad news again) in order to get some of the more sophisticated reviews (from actual medievalists) I had to go ask my local librarian.  

Sometimes research still takes going to the library.  I went, talked with the reference librarians and learned a great deal.  

Thanks, librarians! 

Search on!

Wednesday Search Challenge (July 21, 2010): Finding authoritative reviews of a book?

For the past week I've been indulging a secret interest of mine--reading histories about times, cultures and places I find interesting.  And I've finally gotten around to reading a book that's been on my list for the past few years, A World Lit Only by Fire, by William Manchester.

It's one of those big, sprawling, superlative-ridden histories of a time (in this case, the Dark Ages and into the Renaissance).

But as I was reading I kept getting this uncomfortable feeling--is this all believable?   How much of this should I take as widely-regarded-as-true by the medievalist community?

In other words, how credible is this book?

The issue of credibility often comes up in the context of web research.  (When you land on a page, how do you know if you should trust it or not?)  But the problem of credibility assessment didn't begin with web pages found through web searches; determining credibility has always been with us. 


So I thought I'd start writing a bit about how to determine credibility, and start with a pretty well-known book (it was on the New York Times best seller list for quite a while).  And.. because it's a book, most people will have an idea about how to determine whether to believe it or not. 


So the search challenge for today is this:  How credible is this book (A World Lit Only by Fire)?  

Or.. more succinctly, Can you find 5 or 6 good, highly regarded, trustworthy reviews of this book by people who are experts in the field?


Note that this isn't a black-or-white issue: your solution might be different than mine (and you'll probably find different reviews than I do)... but I want to start a conversation about how to find credible resources. 


This puts us solidly into the land of the reference librarian, but that's just fine with me.  Reference librarians know a bunch of things that we can't afford to ignore--we can learn a great deal from them.


Search on!

Poignant Searches

In the right circumstances, people will admit to the most remarkable things.  They’ll open up and reveal deep, inner truths that they couldn’t tell to their closest friends.  Secret thoughts and desires, not all fully understood, will—on occasion—come bubbling out from within.  Google is, for better or worse, probably the place where most people open themselves up more than any other.  They’ll query for things they’d never dare say aloud; they’ll peek under the corners of their psyche and glimpse what lies within. 

This reminds me a bit of Joe Weizenbaum’s program, ELIZA, which emulated a Rogerian-style psychiatrist through the engine of fairly simple natural language processing.  In his book “Computer Power and Human Reason” (1976), he notes how people would be pulled into a conversation with the program, and reveal—to ELIZA, and only to ELIZA—all kinds of things they couldn’t talk about with anyone else.  The subtle surprise is that people would reveal things to a program that they couldn’t talk about, even with their own analyst.  There was something about being online, about being in an apparently secure and anonymous context that let them release their innermost thoughts. 

By the same token, people ask Google some remarkable questions.   Here are a few that I’ve seen in people’s search histories.  (Comment: The queries I show here are ones I’ve seen in my conversations with people about their searching. I’m not violating any confidentiality agreements here.  As I said, it's remarkable what people will share with you...)

The questions range from the prosaic

     [ why do toenails split ]    

to the historic

     [ why did lil wayne go to jail ]     

with some questions that are practical

     [ why do pdf attachments in mac sometimes display and sometimes not ]

some that are philosophical

     [ compare school education of rich and poor ]

and others that are, for lack of a better word, poignant: 

     [ why do guys who are supposed to like you act mean to you ]. 

At the beginning I was astonished at the range of questions people would ask of Google.  But as John Batelle points out in his book (“The Search”), “Google had more than its finger on the pulse of the culture, but had jacked directly into the culture’s nervous systems.” 

More than just jacking into the culture as a whole, Google seems to be an open, infinitely non-judgmental channel into which we can pour ourselves as individuals as well.  And when we do, we find it rewarding. 

Not only does Google listen, but it also confirms and affirms—your innermost questions, thoughts and desires aren’t as strange as you thought.  There really ARE lots of other people who wonder

     [ why am I so tired ]

or

     [ what sex positions help you lose weight ]

or 

     [ what does white powder coming from a hole in my leg mean? ]

and my favorite... 

     [ how do you fix a goose's broken beak ]

The querystream holds the questions, the clickstream shows the pursuit after the question is asked.  We feel free to ask questions, even though they might be improbably or  impossibly difficult:

     [ why does my boyfriend not want to spend the night ]

What is it they’re looking for? What is it they’re really asking?  Questions like this aren’t really an attempt to find the answer in their particular case, but as a way to understand the possibilities—a way to come up with possible explanations for what’s going on. 

In many cases, for many of the personal and private search tasks, the answer isn’t in any one place, but it’s everywhere.  A query like

     [ is it normal to have hot flashes when you’re 45? ]

might not have a single perfect page with the single perfect answer.  The mere fact that you’ll discover a bunch of other people asking the same question is, in itself, a validation of your question.  You really aren’t alone in a callous universe.  Other people are also asking and wondering the same thing. 

Thing is, when I used to teach Artificial Intelligence at the graduate level, I’d assign my students the programming problem of implementing ELIZA, just so they’d understand the first-hand implications of pattern-matching and creating interesting behavior from a simple rule-system.  As part of their assignment, they’d have to turn in a dialog that they’d held with their program to demonstrate that it worked. 

But it sometimes turned into an uncomfortable grading assignment as I read through their homework.  The conversations often felt a bit too… well… intimate for mere homework.  Most students would do just a short conversation to illustrate the key points of their program—but then there would be a few that would turn in long, extended conversations about their family, their boyfriend or girlfriend, their preferences and dislikes.  And this is just what they turned in—more than a few students told me anecdotally that they spent far longer on the assignment than they’d planned.  They were pulled into the conversations, even with a machine of their own devise. 

Searchers…. that’s what people are.  A homebuilt ELIZA is one thing, but to many, the brilliance of Google is that is in an oracle—a fount of knowledge giving them an infinite number of chances to ask whatever question they’d like, on whatever topic they want to ask… and they can feel completely free to do so.  Asking and clicking feels conversational and very un-judged.

For our collective sanity, I hope it  remains forever thus—a private, anonymous, ever-listening quiet place to seek out the collected thoughts of the world—a place to not just learn about the exotic, the erotic and the eclectic, but also to feel as though you’re part of a larger universe.  Although we all SAY to our students and our kids “there’s no such thing as a dumb or inappropriate question”—it’s just not true.  Can you really  ask the questions that might reveal you as an odd duck, embarassingly dumb or potentially perverted beyond redemption?  Not in any real world with parents, teachers and side-effects of questions. 

Aside from the Google search box, where else CAN you really ask literally any question that you’d like without judgment or loss of face? 

Not anywhere… and that makes the search query box a  very rare place in the world.  

Answer: How to find a list of all an author's books

Questions like this ("find all the items of type X") are always pretty hard unless you know the set is a closed and well-known.  If you're looking for all the US National Parks, then it's pretty easy to find a complete list.  For prolific authors like Rheingold, even HE might not have an authoritative list of all the books he's written.


(This isn't as crazy as it sounds.  I've written an awful lot of scholarly articles--but I'm not sure that one or two haven't managed to slip off of my CV sometime during the last couple of years.)


In any case, how WOULD you find a complete listing of an author's works?


Alas, there's no single answer for something as complex as a complete listing of an author's published books  You're going to have to work for it.  Here's what I did...


1.  Check the author's website.  (In Rheingold's case, it's www.Rheingold.com  As you'll quickly see when you visit, there's no immediately obvious listing of all his books there.  He shows "selected" publications only, which is common enough among well-published authors.  


2.  Check the author's Wikipedia entry.    Same story here--again, it's incomplete.  http://en.wikipedia.org/wiki/Howard_Rheingold


3.  Check WorldCat.  If you don't know about WorldCat, you probably should.  It's a metaindex of many (many!) of the world's libraries.  They aggregate the catalogs of more than 10,000 libraries worldwide, giving you the ability to search for things like "Books" by author "Rheingold, Howard."  Their Advanced Search interface is especially handy for searching and filtering by specific metadata such as Author, ISBN number, publication format (book, CD, DVD, language), type of publication (fiction, nonfiction, documentary.  And so on.  When I do my search on WorldCat for au:Rheingold, Howard (selecting on Books in English), I find "about 50" hits... but not all of them are really authored by Rheingold, we have to do a little quality control.  (More on this in a minute.)  


4. Check Amazon.  You probably know about Amazon, the huge online retailer and book supplier.  They also have a pretty complete listing of an author's works.  But since they're focused on book sales, they make no claims about how complete their listing is.    


5.  Look for book specific databases.  For example, many libraries provide access to "Contemporary Authors Online" (a database from Gale, a part of Cengage Learning).  This is a fairly complete listing of contemporary authors' works.  Another resource to consider would be the Books In Print database (noting that you can also search for books "out of print" and "forthcoming" as well.)  


6. Check the Library of Congress.  The "nation's library" is easily accessible through LOC.gov and you can search for books by author.   


7.  Check Google Books.  If you use Advanced Search on Google Books, you can search by Author, filtering by publication type (in this case, Books), and set the "number of results" to 100 (the max) in order to see all of the published books.  In Rheingold's case, Books finds 50 titles as well (although some are in non-English languages -- there's currently no way to filter by book language).  

One thing that's interesting here is that all of the resources have very different lists!  What the author lists as his books isn't the same list as what Wikipedia or WorldCat or Amazon or Google Books lists.  

So, how do we get to a decently authoritative list?  

Here's what I did.  I took each of the resources and put the list of books they gave me into a spreadsheet.  (For instance, you can have the LOC email you a list of all the hits they found, and you can ask WorldCat to create a single list of all your hits, which you can then cut-and-paste into your spreadsheet.)  

After a bit of mucking about to get all of the data into a spreadsheet where the first column is all titles, I just sort by title, then filter out duplicates.  

Turns out this didn't QUITE work, as there are some variant title names (and typos!) in the lists.  But it's not intractable to just go through and fix everything up by hand.  (We're talking about a single author's output, not all of the books published in English during the past decade, which would be a real data-cleaning problem.)  

Now, what REALLY made this search task interesting is that Howard Rheingold is one of those people who's on the verge of being a brand.  That's why some books have "Howard Rheingold" in the title (e.g., Whole Earth: World Wide Web, Buckminster Fuller, Wired, Whole Earth Catalog, Stewart Brand, the Well, Kevin Kelly, Howard Rheingold, by an outfit called Books, LLC.  I'm sure there's a completely plausible story behind this, but it makes determining an authoritative list a bit tricky.)  

Then there's the range of what Rheingold has done--is an audio cassette of an interview with Rheingold the same as a published work by him?  What about compendia (such as the Whole Earth Catalog) which are published multiple times in different editions?  

This is the kind of thing that professional librarian catalogers deal with on an everyday basis.  But for us who are just plain searching for a master list, makes you realize why the world isn't as simple as you might think on first blush.  

In any case, after collected the lists from the different resources, I'm happy to finally have a list that I believe is complete and accurate.  (For my working definition of a book written by or edited by Howard Rheingold.) 

Listed in alphabetical order:  

  Ace it!: use your computer to improve your grades

  Ars Electronica 2003: Code:The Language of our Time 
  Excursions to the Far Side of the Mind:A Book of Memes
  Exploring the world of lucid dreaming
  Higher Creativity: Liberating the Unconscious for Breakthrough Insights
  Open 11: Hybrid Space
  Out of the Inner Circle a Hacker's Guide to Computer Security
  Silicon Valley Guide to Financial Success in Software
  Smart mobs : the next social revolution
  Stereogram
  Talking Tech: A Conversational Guide to Science and Technology
  The Cognitive Connection: Thought and Language in Man and Machine
  The Millenium Whole Earth Catalog  (1994)
  The Millennium Whole Earth Catalog: Access to Tools and Ideas for the Twenty-First Century
  The New Technology Coloring Book
  The Savage Report 1994, Jack Anderson Against Dr Tek 
  The Virtual Community: Homesteading on the Electronic Frontier 
  There's a Word for It They have a word for it : a lighthearted lexicon of untranslatable words and phrases  
  They have a word for it II : a lighthearted lexicon of untranslatable words and phrases (the Writer's Studio
  Tools for Thought: The History and Future of Mind-Expanding Technology
  Virtual Landscape/Howard Rheingold: New Technologies for the New Millennium 
  Virtual Reality - The Revolutionary Technology Of Computer Generated Artificial Worlds
          & How It Promises To Transform Society
  War of the gurus


That's quite an output over the past few decades.

And it makes my point really well--even "simple" search tasks often require some non-obvious work and manipulation to get to a reasonable solution.  Here, it was looking for multiple sources, then integrating the results.  Use the tools you've got (email, copy/paste from web page, spreadsheets..) to assemble a non-redundant list from multiple sources.  I can't think of any other way to do this.

Can you?

Search on!

Search challenge (Wednesday, July 14, 2010: How to find a list of....

One of the critical steps in sensemaking is often collecting information about a given topic.  The search problem here is that collections on your topic-of-interest often don't exist.


And that's a challenge.


Suppose you're interested in different kinds of flower pollen, or drum and bugle corps in California, or 18th century Spanish missions in the new world.  Whatever the topic, you usually start by collecting and organizing information on that topic.  And, depending on how scholarly (or compulsive!) you are, you might delve into original research (say, travelling to Spain to look in the archives for evidence about those missions)... or you might just be happy with collections that other people have put together.


In any case, that collection phase is an important one.  And it's often the step that stops many people.  Short of laborious work, how can you find 20 different examples of microphotographs of pollen?


While that's an interesting question, that's not the challenge for today.  (For that particular collection, I'd probably use Google Images, or look at the Wikipedia article on pollen.) 


Instead, today's challenge is this:  


Can you find a list of the books for which Howard Rheingold is the author OR co-author?   


You could substitute any other well-known author for Rheingold here, but I want to make the challenge interesting. (Besides, Howard has created an extensive and fascinating oeuvre covering a wide range of topics.  He's one of the few writers I know to have such a broad Dewey Decimal range!)   


And, for simplicity's sake, let's only look for the original English language editions.

Knowing what you can search for...

One of the things that limits people the most in their searching is simple:  They can't imagine searching for it in the first place.

This sounds too weird to be true.  I mean, can't you just imagine it, and then search for it?

The answer is mostly not.

Recently there was the tale of the long-lost letter of Descartes showing up in the Haverford College library.  It's a heartwarming story of a letter that was stolen in the 19th century from a European library, then making its way to a small Pennsylvanian liberal arts college, to rest quietly in their library.

Last year, a Dutch scholar, knowing of its existence, decided to do a Google search for the letter.  Lo & behold, search returned a result for a catalog entry in the Haverford Macgill library.

It's a good story... but it makes the point:  If you don't think to search for something, you're never going to find it!

Alas, we're hemmed in by our cognitive mind sets more than we're limited by the capabilities of search.  As the Descartes letter incident shows, just because a document isn't online doesn't mean you can't find it.

For instance, one of my favorite books is Dream Machines: New freedoms through computer screens--a minority report. (Self-published book by Theodor H. Nelson, 1974) It isn't easily available, although you can buy a used copy on Amazon for $95.  And many libraries just don't have a copy.

But it's pretty simple to find via Google Books.  Just do a search on Google Books, then click on the "Find in a library."

And I quickly discover that it's at Stanford, just a couple of miles away.  (Google can do this by handing off the book information to WorldCat, which is a meta-catalog of many libraries around the world.)  

I've written about "Find in a library" before, but this time I want to emphasize that if you don't think to look, you'll never find.  Don't let your imagination limit what you might try to search for.  

My current favorite:  [ why is a raven like a writing desk ] 

(I like it because it's a riddle from Alice in Wonderland, and the story written on this by Cecil Adams of The Straight Dope, is wonderful.)  

Answer: What's this mysterious rock wall?

Here's the quick answer... It's a slickenside.  


How did I find this?  This wasn't a terribly hard problem.  Here's how I approached it.

My initial query:

[ polished rock wall San Francisco ] 

got me the street address (on Beaver Street, at the Corona Heights playground).

At this point I could use the Reverse Dictionary trick (see my earlier blog post), and that would work pretty well.    Just go to a reverse dictionary such as One Look and do a query there on [ polished rock wall ] -- the second result is "slickenside," which is the correct answer.

But to do a regular Google search, I had to cast around a little bit until I found the query:

[ smooth polished rock surface ] 

At which point I noticed the strange word "slickenside" -- when I read that article comparing glacial striations with polished rock surfaces, I quickly realized that this was exactly what I'd been seeing... right down to the parallel lines on the face of the rock.

To confirm this, however, I had to do one more search:  I wanted to find the term "slickenside" used to describe the rock wall at the Corona Heights park.  My verification query was

[ slickenside Corona Heights ] 

which led me to a very nice article about the slickenside at the playground in the Corona Heights playground.

So there you go.  Two different ways to approach this problem.  The Reverse Dictionary method and finding another way to say "polished rock wall."

They both work just fine.

Search On!

(And if you'd like to visit the slickenside in SF -- here's the Google Map link.)

Wednesday search challenge (July 7, 2010): What's that mysterious rock wall?

A true story... 


I was visiting San Francisco and had a bit of extra time.  Near the corner of Market and Castro I started walking around the neighborhood, just to see what was there. 


I discovered a most amazing little park.  It's tucked away in an obscure corner of the city, and it's dominated by a large, smoothly polished wall of rock.  You sort of have to see it to believe it, but here are a few photos. 


This is a section of the wall.  The entire exposed wall is probably 100 feet long, and runs from ground level in the playground up about 50 or 60 feet, when it slopes away, turning into just an ordinary hillside.  

Here's another, closer image.  You can see how smooth the rock wall is polished--it even reflects the color of the sky here.  

A last picture, just so you can be as amazed as I was.

So... what's the question?  I want to learn more about this thing--but how?   


This is clearly some kind of really interesting geological feature, and--darn it--I didn't have any geologists with me.  I DO know that this rock is a kind of chert, and the rock face is highly polished.  Maybe that will help you in searching.  


What's the specific word (or search term) you want to find out more about the geology of.. this polished rock wall?  


What's it called?  


And... for extra credit, what's the address of this wall? 

Answer - What's happening with employment in California?

When I asked "What's happening with employment in California?" I was purposefully framing the question in a way that someone might think of it.  That is--I did NOT prime you with the search term [ unemployment ].  


If, however, you managed to see your way through that obstacle, you might have done a query like this:  


[ California unemployment rate ] 


Which would bring up a Google shortcut result in position one, like this:  

This is the "Public data" result, which leads you into a marvelous compilation of public data that's been collected and organized by the same Google team that's brought you the bubble charts in Google Spreadsheets.  


From our perspective, though, it's a single-stop for comparing unemployment rates by county within California (or any other state).  In particular, we can simply answer the question I asked earlier (that is, compare the employment rates between Santa Clara and Santa Cruz counties).  


With three mouse clicks, you can select the data from Santa Clara and Santa Cruz (and deselect all of California) to produce this graph: 

(Teachers:  A good question to ask is "Why is Santa Cruz county so much more variable than Santa Clara county?"  The answer has to do with what each county does economically--Santa Cruz is much more agricultural, hence the seasonal swings as ag-workers are employed or unemployed.)  


Note that you can hover over a particular data point to drill-down to specific dates. 

In this particular case, the data is supplied by the US Bureau of Labor Statistics (which you can discover by clicking on the link in the lower left of the graph).  As always, you want to verify that the data comes from a reputable source.  


Is this just a trick?  I've had some people complain to me in the past that queries like this are "just tricks... there's nothing methodical about it, you just have to know a lot of stuff." 


And, to a certain extent, that's true.  But this is the nature of expertise--to be a real expert at something takes practice and knowledge.  The wonderful thing about reference librarians is that they already know a great deal about how the information in the world is organized.  They know what resources exist and what's possible to do with them.  


My deeper point in this post is to recognize that sometimes you can stumble across resources (such as the Public Data shortcut I've shown you here).  When this happens, take note. Knowing where the data is kept and how to access it really is a fairly important part of becoming a great researcher.  
_________

You can learn more about the range of shortcut results at the Google Helpcenter page Explore Google Search.

And, to learn more about the other kinds of public data that's organized by Google, see the Public Data Explorer site, which will connect you to data about World Economic Development, birth rates, unemployment (US and EU), cancer causes, and many other wonderful data sets.

Special note for teachers:  These datasets are great for teaching concepts about statistical analysis.  

Wednesday Search Challenge (July 1, 2010): What's happening with employment in California?

As we've discussed, one of the more difficult search tasks is to find data sets that you can use.  After all, it's one thing to look at published reports about the oil spill in the Gulf, or the loss of migratory bird species, or the changes in the polar ice caps--but it's really much more compelling if you can examine the data first hand.  


If you think about it, grappling directly with data removes at least one layer of interpretation.  To understand a technical topic, it's often truly illuminating to do your own analysis.  


The key problem is, usually, where do you find the data?  


And that's the problem for today. 


 Can you find a data set that shows (by county) the out-of-work rates for California?  


Then, once you've found that, can you create a chart showing the relative rate of change for Santa Clara vs. Santa Cruz counties?  




(And yes, I know today isn't Wednesday.  I was off at the ISTE conference for the past couple of days.  More about that next week.)