Monday, October 5, 2015

Answer: Thinking outside the box


When I say... 

"Thinking outside the box," I could be referring to any of many boxes that we get ourselves stuck in.  As you know, there are lots of bias effects that can influence our problem-solving (or research) skills.  We've talked about some of them before:  confirmation bias (finding information that confirms one's beliefs while giving less attention to information that contradicts it), the availability heuristic (counting on immediate examples that come to mind when considering a specific topic), and functional fixedness (using something only in the way in which is typically described or thought-about).  

There is an important variation on these biases.  Let's call it the common reference effect.  It is the tendency for people to look first (and often, only) at the reference material that they commonly use.  It's a little like the availability heuristic in that it's an effect of "using what you already know about," but it's a bit more than that.  

The common reference effect is when people tend to use the same reference materials repeatedly, even when it's more difficult to use or less-complete than another source.  

Our Challenge questions for this week are: 

1.  Can you create a chart showing the difference in the populations between North and South Korea since 1970?  (Just a simple line graph would be fine, thanks.) 
2.  Can you compute the market cap, total revenue, and number of outstanding shares for each of the companies IBM, Apple, Google, and Xerox?  
3.  Having recently dived in the Caribbean, I'm really interested in whale sharks.  Can you quickly compare blue whales, gray whales, sperm whales, and whale sharks in terms of (a) lifespan, (b) maximum length, (c) weight?  (Just the facts, ma'am.)  

As several readers pointed out, yes, you can search on Google for these questions.  For instance, for the Koreas population question the simple query of  [ population South Korea ] will give you a lovely chart that also includes North Korea.  




The second chart is especially useful because it gives the date of last update to the data (July 27, 2015) and the source (World Bank), and you can do various analyses with the data (e.g., compare to other countries or data sets).  

But it doesn't quite give you the difference between the two populations.  

Likewise, it's straightforward to use Google to get to the financial data for each of those companies.  Just do a query for the NASDAQ stock ticker name, and you'll get something like this: 


A few more searches will get you the rest of the data (market cap, etc.). 

And naturally, if you do a search on Google for [ blue whale ] you'll get a bunch of great results, and a nice Knowledge Panel in the upper left telling you more about this particular whale (e.g., the largest animal to have ever lived--bigger than the biggest dinosaurs). 


And while you can do SOME side-by-side comparisons on Google, you can't do arbitrary comparisons.  Here's one you can do: 


NOTE that the light gray downward arrow is a clickable target!!!!  If you click on that, you'll get a bunch of additional information about your two comparison items.  (See next image.)  



HOWEVER...  it's not possible (currently) to compare any two arbitrary items side-by-side on Google, even when they're similar kinds of things.  (For instance, you can compare [ avocado vs. salsa ] but, oddly, not [ guacamole vs. salsa ] which you might think you could.)  

But as several regular SRS readers noted, the Wolfram Alpha search engine can do this fairly easily.  

In general, as SRS-ers, we want to be aware of what tools exist, and what their special and unique capabilities are.  And one of the things that Alpha does quite well is side-by-side comparisons.  

So if we do that same [ guacamole vs. salsa ] query on Alpha, we see a useful comparison: 



But notice something here:  See the arrows?  You have to be a little careful here--the amounts being compared are "one serving" (which can be very different).  When you look at the rest of the chart below, be sure to remember that these charts compare 2 Tbsp of guacamole (28 grams) with 1/4 cup of salsa (65 grams... nearly twice the amount of guacamole!).  

I don't know about you, but I eat these in about the same amounts.  So if I ate 1/4 cup of guacamole, I'd really be getting twice the number of calories shown here.  (In general, be very wary of "serving sizes"--they sometimes don't match real experience.)  




In any case, from a SearchResearch perspective, this is the kind of tool we need to find these tables of comparisons.  Alpha does a pretty good job here.  

Korea population:   

Notice how I framed the query here--as a math expression.  Alpha does a great job converting arbitrary inputs into mathematically meaningful expressions.  



Notice in particular the chart is the DIFFERENCE between the two countries, and shows the change over time.  

An issue here is figuring out where the data actually comes from.  If you scroll to the bottom, you can click on the Sources link, which brings up a particularly unhelpful list, including that the "Primary Source" is Wolfram|Alpha Knowledgebase, 2015.  That's a bit like saying "Because I said so..."  

There is a pretty generic list below that of sources that the Knowledgebase draws upon, but it's pretty vague.  

(Apparently you can send them a message and ask for details, but I haven't tried that yet.)  




Financials:  How would you do something like this analysis in Alpha? 

Just copying the string of company names into Alpha gives you this... 


Again, read the fine print carefully.  See that "Assuming GOOGL"?  (Pointed to by the red arrow.)  Be aware that when things like that show up (or in the Google results way above), they can be important.  Google stock is now listed under TWO stickers:  GOOG and GOOGL.  To get an accurate picture, you need to consider both... notice where the values are the same, and where they differ.  





Sea Creature Comparison:  Now we know that Alpha is good on demographics and financials--what about biology?   Do a similar kind of search... 



As you see, the side-by-side comparison tables are fairly interesting, and certain cover the lifespan, length, and weight we were interested in. 

The aspects of the tables can be somewhat random (e.g., "length of whole intestine"), but the capability of automatically generating side-by-side comparisons of different kinds of things can prove incredibly useful.  

For instance, suppose you've been reading about VW cars recently (just, hypothetically) and you'd like to get a quick sense of how two different VW cars differ.  Here's an example: 





Search Lessons:  


Tools:  As always, know your tools.  Among its many capabilities, Alpha is GREAT about generating tables of side-by-sides comparing and contrasting different kinds of items.  

Verify:  On the other hand, verify-verify-verify.  If your task is critical, then you'll want to double-check the actual values presented in the tables.  (But that's true for all search engines...)  

Check the details:  And always read the fine print.  I've seen lots of searchers get tripped up by doing a search and landing on a chart or graph that seems at first blush to confirm what they're looking for.  CAUTION!  It's easy to find things that confirm your bias and your expectations.  Whenever you find a chart / data, look for the source, and check all of the tiny print around the outside.  Be sure you know what assumptions are baked into the analysis.  (For instance, how much of guacamole is being considered in the analysis?  A factor of 2 can be deeply important for your next Mexican-themed party!)  

Common Reference Effect:  Don't get trapped by the common reference effect.  Great researchers know multiple ways to find information, and know what the strengths (and weaknesses) are of all of their data sources and tools.  

This was a fun Challenge!  Thanks to everyone for their contributions!  


NOTE:  This week's Challenge will come out tomorrow, on Tuesday rather than Wednesday.  (I'm traveling for much of this week, so I need to push it out tomorrow.  Happy searching.)  





4 comments:

  1. This was for me one of the best challenges and search lessons ever. I *know* I am frequently trapped by this Common Reference Effect. I didn't have a name for it but I knew all along that sometimes people find results faster than me even when they're less proficient at the generic task of researching on the Web, just because they use a tool I disregard in favor of the "always reliable" Google tools. Also, I have the absurd tendency to value answers that take me some time and work, in a sort of IKEA effect (placing too high a value on some result I "assembled" myself). This is related to my procrastination tendencies and my close-to-obsessive love for thoroughness, as well as maybe being a kind of NIH syndrome. This often results in a planning fallacy (an underestimation of the time I need to complete any task).

    As if I had to prove myself or anyone else my love for thoroughness, I have just read the entire list of cognitive biases on Wikipedia — as someone may already have guessed from last paragraph's bias-name-dropping… I found that there is a so-called familiarity principle (aka mere-exposure effect) which the now coined common reference effect seems to be a corollary of. Also, the common reference effect may be related to the well travelled road effect.

    All in all, I do believe that my failure at this task was mainly due to, plainly said, lack of intelligence (or of the particular sort of intelligence needed here). Even though I had used Wolfram Alpha some times, I wasn't intelligent enough to fully understand under which conditions it is a useful tool, until someone who had been able to summarize that taught me. Fortunately, I also believe I can train and improve my intelligences. And if anyone says that's not possible, I'll be glad to be stubborn and keep practicing my confirmation bias on this. ��

    ReplyDelete
    Replies
    1. Nicely put, Luís! This is one of the big points of reading SRS -- we all learn useful skills and bits of knowledge that make us more effective searchers...

      Delete
    2. I agree with Luis and Dr. Russell. Some are great for some kind of challenges and others in different one. That is why is always good to read others comments; always great knowledge in them.

      I also like this Challenge a lot because was totally new tool. And because, at least I can't, finding the tool without knowing it is hard. I mean, there is no query that before the Challenge will give answer W|A. Now, there is. Dr. Russell's Challenge is in Google so when someone looks for something similar, they will learn too.

      Common Reference Effect : Guilty too. Check the details and Verify are great lessons, it is good that only few times only one source in the data.

      Remmij, I like the photos you tried, in special doughnuts. I thought that should be something system could identify without problems. Cryptids, there are plenty of them, thanks for sharing the link. I tried with some like Scooby Doo (fail) dogs (gave breed), words (fail). However, I think is a good tool to know.

      Have great week everyone, and enjoy trip, Dr. Russell.

      Delete
  2. …useful reminders… now where did I leave those avocados
    …not quite a chip dip, but a reasonable guac substitute…
    Nutrition information per 1/4 cup: 95 calories (4tbs/¼ cup)
    MangoMole
    AvsM
    fruitvs
    pasilla
    chilaca
    dip list
    a form of a cognitive bias
    incomplete list

    ReplyDelete