Thursday, June 7, 2018

Survey Results--an analysis of advice about searching


Last week we had a survey.  

It asked the SRS community about what they thought were the most important search skills.  (Take it by clicking here, if you want to see what the original survey was all about.)  

Since I do surveys professionally (for my research work at Google), I know this isn't a perfect survey.  The sample size is too small, too biased towards professional researchers, and doesn't have a broad enough set of questions to be accurate. 

I don't care.  

What I wanted from this survey is a sense of what experts think about search advice.  That is, I really wanted to hear what you, gentle reader, had to say about search methods and conducting online research.  

The good news (for me) is that many of the ideas you put into the survey are covered in my book.  The better news is that I don't cover everything you mentioned.  

To everyone who filled out the survey, many thanks for your comments.  They were thoughtful and had great insights, including a few that I hadn't thought about before!  We'll cover some of these in future posts.  


Analysis:  We got 48 responses from readers.  I was hoping for a bit more, but this is a great foundation to start.  There were also 4 questions, so that's 192 responses that I'm summarizing.  

All I did was to read through all of the responses and summarize the themes I saw.  Each of the items below was suggested by more than one person.  The rank order reflects the number of times each was written about.  #1 in the list was the most common, #2 was second most common, etc.  

Here's what you said, with a short explanation / summary afterwards. 


Most important skills:

1. Query formulation (and reformulation)
This was by far the most common skill that you think good searchers should have.  This makes sense, as the quality of your search query is a strong determiner of how successful your search will be.  Over the years I've seen that some people get stuck when they can't figure out the right search terms--that's an important barrier for getting to a successful answer.  Reformulating your query is important, both to get out of being stuck, and to hone in on what you're actually seeking.  (More on this below.) 
2. Learn from previous searches
We've talked about this in lots of SRS posts--a very important skill is that of reading and learning from your search experience.  In classes, too often I see people not reading the results page and consequently NOT learning what worked or didn't work.  Careful reading and thinking about what your search returns leads to learning, and learning leads to better searching in the future.  

3. Know what’s possible
Knowing what you can search for, and understanding the ways in which you can search, is absolutely fundamental.  In my testing of search skills, one of the things that completely blocks people from completing their search task is not knowing that you can search for X, Y, or Z (you fill in the blanks). In one dramatic example, 250 Google engineers were unable to complete a search challenge I gave them because they didn't know it was possible to search for archival aerial images on Google Earth.  (Did you?)  

4. Context
When you search for something that's more than just a simple answer, you often need to find the context that surrounds what you find.  This is especially true for complex topics such as social or historical questions.  Just reading a single web page (or, God forbid, reading only the snippets on the SERP) almost never gives you enough context information on the topic.  Read widely, and learn the when/where/why/how about your topic.

5. Lateral searching (using tabs and windows to organize your searching)
I'll be writing much more about this in weeks to come.  My colleagues Sam Wineburg and Sarah McGrew (Stanford) have written really well about this and why you'd want to do lateral searching. See: Lateral Reading (Sept, 2017).  I'll be writing about some effective strategies for doing this.  

6. How to limit your searches to a particular domain 
Limiting the scope of your searching is simple, if you know how to use site: or filetype: -- those are the simplest ways to get results with certain kinds of properties.  You use site: when you want results from a particular web site or organization.  Thus, [site:CDC.gov influenza] will give you results with a distinctly CDC perspective.  Likewise, you can exclude certain sites with the minus operator.  [-site:CDC.gov influenza ] Or you can search multiple sites or entire domains:  [site:.IN Brexit OR site:.BE Brexit] will search for "Brexit" on Indian or Belgian sites. 

7. Ability to evaluate what you find
Yes!  This is critical... and requires much more discussion than I have room for here.  We'll return to evaluation rules-of-thumb in future blog posts. 

8. How to find experts and learn what they know
This is a great way to find advanced, expert content on a topic.  The trick here is to locate experts that are truly expert (why do you think they're an expert?) and then tracking down their writing.  The other trick here is to always locate more than one expert.  Ideally, you want to see the varieties of thought on a topic (there could be more than one point-of-view... in some cases, there can be 4 or 5!), and get experts from each perspective.  

9. Critical thinking 
A critical thinker is always wondering how could this be wrong?  And how can I break this big problem down into pieces? There are many approaches to critical thinking, but these two heuristics work well for me.  A critical thinker criticizes an idea, some writing, or a point-of-view... but does so productively, not to just be mean-spirited about it.  Critical analysis is trying to break something into its components to understand how it works.  It does not assume that authorities are always correct, but calls things into question.  (This is why critical thinking annoys some people.)  Of course, you want to do your critical thinking with care, and not be annoying about it.  But it's an essential skill.  



Most important attitudes:

1. Persistence

Persistence in the face of failure is the #1 attitude (or trait) that was mentioned by those surveyed.  I agree.  It's obvious in my search classes when someone has this attitude, and when they don't.  Much of what I teach is attitude encouragement.  "If that didn't work, what do you think would work?"  And "Just try one more search.."  

2. Curiosity 

You'd think this would be obvious, but I agree.  Intrinsic curiosity about the world is essential.  On the other hand, I don't know how to communicate this other than by being curious myself and showing what kinds of things you can learn about the world through satisfying your curiosity.  (In many ways, my book is exactly this--ways to find answers to your questions about curious things.)  

3. Adaptability 

Adaptability is just the ability to respond to changing circumstances.  In online research, it's the ability to act effectively when things go wrong (e.g., getting 404 errors, finding that your favorite website no longer supports that service, etc.)  You'll also find that you need to adapt to changing tools (new features introduced, old ones go away) and data.  

4. Enthusiastic

 Someone's enthusiasm goes a long way in making both persistence and curiosity work.  I've seen people who were not great searchers manage to succeed because they had great enthusiasm for their topic, which in turn made them persistent, curious, and adaptable.  (These people are always great to work with.  Their enthusiasm for the topic means they want to learn how to be better and more accurate.)  


How to ask good questions:  

These answers were pretty interesting, and all over the map.  Here's my condensation of the top 10 ideas. 


1. Be specific / be clear about what you’re asking

2. Ask yourself:  WWDD  (what would Dan do?) 
I was surprised about how many people said this.  Aww.. gee... thanks! 

3. Who would know the answer to your question… how would they put that info online?
This is a variation on the "find an expert" heuristic from above.  But I like it as a heuristic for asking questions too.  How would an expert think about your question?  

4. Think of alternative ways to ask the question

5. Start large – don’t zero in on a specific topic

6. When stuck, work from specific examples

7. Always ask why? 
That is, ask why about the answers you’re finding.  Why is this true?  And How do you know?  This almost always leads to a better question. 

8. “Predict the answer”
One way to create a question is to think forward to what an answer would look like.  That is, if the answer looks like this, what question would let me get to that?

9. Ask yourself WHY do you want to ask this question.
This is a classic library reference desk question.  A great reference person always tries to dig into what you want to know.. and why.  Knowing the answer to why tells you a lot about how to frame your question.  

10. Be open to nuance. 
Don't give up on the subtlties of a question.  You can almost always ask "Is there more I should know?"  That's a useful question in almost all cases. 



Other advice from the survey

1. be aware of other search tools (e.g., thesaurus, dictionaries, geographical tools, …)

2. don’t limit yourself to English and US sites.

3. search for tools to help you accomplish your task

4. recognize that Google doesn’t have everything

5. be humble in your searching

6. consider the motivation for whoever is writing whatever you’re reading. Why did they write this?  Why did they post it?

7. be aware of confirmation bias


Summary 

There's obviously a lot that's been packed into these answers.  I'll be unpacking some of these themes over time.  

You'll be amused to know that the attitudes section is almost exactly what I cover in chapter 19 of my book.  Thanks for confirming my intuitions!  

See you next week when we discuss the answer to the mysterious avocado seed question! 

Search on.  


Wednesday, June 6, 2018

SearchResearch Challenge (6/6/18): Seed dispersal mechanisms for giant seeds? (And search strategies)



I'm back! 

Welcome to the new post-book SearchResearch.  Just so you'll know, it's provisionally called The Joy of Finding Out.  

A quick book update:  I finished the first draft of the manuscript.  Now comes the copyediting, approvals, publisher comments, figuring out the color pages, etc etc.  One of my friends who's an accomplished author said "Congratulations.. now the work begins!"  

Ummm.... What was all that stuff I just did? 

When this thing is all done, I'll give you a bit of a breakdown in terms of effort--how much time was spent writing, how much time editing, etc.  

Realistically, this book is probably a 2019 publication.  (But I'm hoping for an earlier, limited release!)  Stay tuned. 

I know I promised you a  breakdown of the survey results from last week, but I'll get to that tomorrow.  There were a LOT of insightful (and lengthy) comments--it's a bit more than I can whip out in just an hour or so.   So I'll give you a new Challenge for this week, and give a summary of the survey tomorrow.  


WHILE I was working on the book, I ate a lot of the millennial's favorite, avocado toast.  Since I live in California, I usually just make my own avo-toast at home, which means I look at a lot of avocados.  


  
As I was tossing one into the compost bin I started to wonder about the seeds and how the avocado seed is dispersed in its natural habitat.  Since I used to live in Los Angeles, I've seen a lot of avocado trees. They get pretty big and can have a thousand avocados on their branches.  But when ripe, they just plummet straight down.  Let me tell you, it can get really messy down there.  

They're pretty easy to grow, but how does an avocado get around?  I mean, are there forests of avocados??  

That led to me today's Challenge: 

1.  How are avocados dispersed?  If they rely on just falling from the tree, that doesn't seem to work well... so is the "natural" dispersal by animal?  If so, WHAT animal would eat an avocado... and then be big enough to carry it somewhere and leave it behind in a new place?    
2.  Are there other plants with giant seeds (like the avocado's) and how do THEY get dispersed?  
3.  In writing this post, I’ve been looking for an early illustration of an avocado.  It's kind of tricky.  What’s a strategy for finding the earliest illustration of something?  In our case, an avocado seed... but how about finding an early illustration of ANY thing?  


Once you figure out the answer, let us know... AND be sure to tell us HOW you figured it out!  

I'm curious to see what your "old illustration" strategies will be.   

Search on!  (And keep eating avocado toast.  The avocado ranchers need your support.) 



Wednesday, May 30, 2018

SearchResearch - update: Progress!


I'm almost done! 

Well... that is, the writing of the first draft is nearly complete.  Only another 2 or 3 thousand words to go.  (That sounds like a lot, but it's an afternoon's worth of writing.  The problem is that a lot of those words then get edited out in a later cycle.  Still... it's progress!)

In this last week before we go back to our regularly scheduled program, I'd like for you to fill out a little survey.  It's your chance to alter the course of the book!  Have a minute?  Contribute your brilliant thoughts to this small survey.  I'll summarize what I learn in next week's post.

Search on! 



Thursday, May 24, 2018

SearchResearch -- Delay of game for 2 weeks


Normally... 

... we'd have a SearchResearch Challenge this week.  BUT... as you know, I'm working on a book about the SearchResearch experience.  It's not about SearchResearch per se, but about what we learn here--how to frame questions and how to use our research skills to answer those questions.  

Well, the update is that I'm nearing the end of Draft 1.  So, for the next 7 days, I'm working on a deadline to get everything done and turned in for first round of editing.  

This has been a tremendously exciting project, and I hope you'll all recognize some of the chapters!  Our blog has been incredibly influential, and as I'm wrapping up the first round of writing, I'm constantly re-finding ideas that we've shared over the past few years--and I appreciate all of your comments.  Thanks for all the ideas and support over the past 8 years!

(Yes, SRS started in May of 2010 with 1032 posts and 2.995M reads.)

PowerSearchingWithGoogle.com logo--almost 4M students have taken the class.
PowerSearching SearchResearch for all!  

So I'm going to take this week off and give you an update next Wednesday.  (But I'll be working harder than ever on SRS!)  

Don't go away.  I'll keep updating you with progress as the book goes forward... and there will be another Challenge next week.  

In the meantime, Keep Searching! 

-- Dan 




Thursday, May 17, 2018

Answer: Swiss Mysteries

This was supposed to be a fun Challenge... 
... but perhaps it was more complex than I'd thought it would be.   

Let's go through each, one at a time... 


1.  When in Switzerland, one naturally eats a lot of chocolate.  (I certainly did.)  But as I was munching on a bar of Lindt milk chocolate, I realized that the image on the bar of chocolate made no sense to me.  WHAT is going on with this cup?  Why is the handle so strange?  (Did Dali do this illustration for them, or what?)  





If you don't know that this is called a "dipper," then it's a lot harder to figure out.  Luckily, I knew that word, and doing an Image search for: 


     [ milk dipper -constellation ] 

pretty quickly finds similar contraptions.  Note that I added in the -constellation to get rid of all the "Big Dipper" constellation images.  



And once you see these "cups with odd handles," you'll learn that they're called dippers.  

Unfortunately, I couldn't find an exact match to the white cup shown on the label... but close enough.  

By the way, I tried doing a subimage search by cropping the logo to just the cup shown, but that only found more bars of chocolate, much to my surprise. Apparently, there ARE no images of the Lindt white milk dipper cup.  (If you find one, let me know!)  

Of course, if English isn't your first language (or if you just can't think of what that word might be), remember that you can always use a reverse dictionary to help find synonyms or a word that fits your semantic notion of what you seek.  

For instance, you can use the OneLook reverse dictionary function like this: 


Here I searched for the phrase "cup used to..." (in the sense of "an X cup is used to...").  I don't know quite what X is, but as you can see, a bunch of great terms show up, most of which you can ignore safely.  (Acetabulum?  Acorn?  Agate?  Probably not.)  But when you get to dipper (number 64), you can do that search with that, and you're on your way.  

And you'll learn that the handle of the dipper is intended both to make it simple to hang onto the edge of a milk pail (or bucket), and the angle keeps it tilted away from touching possible contaminating surfaces.  



2.  One day I walked into a bakery in central Zürich and spying this delectable pastry, I ordered a croissant.  I was very surprised when the clerk said, "I'm sorry, we don't have any croissants."  But this is what I saw in the display case:  





Obviously, I successfully ordered this item. What SHOULD I have called this thing that looks-like-a-croissant?  When in Switzerland, what are these called?  
  
This surprised me when I first saw them in Zürich, but a quick search for: 

     [ Switzerland croissant ] 

taught me that they're called gipfeli in Switzerland, and as you probably found, there's quite a bit of back-and-forth about whether they're worse (or better) than traditional French croissants.  But then again, there's a fair bit of debate about the origins of the croissant as well.  When in Switzerland... 

3.  I understand more German than I speak, but every so often I would hear someone say something odd.  One construct I heard that seemed odd always involved people's names.  For instance, "Wo ist der Hans?"  Can you figure out why that sounds odd to me (a native English speaker), and why the word "der" is in that sentence? 

When I did this research, I knew that the word "der" is the definite article, so my first search was for: 

     [ definite article German names ] 

which took me to this fascinating article on StackExchange (which is often a very high-quality source for information about technical topics, including language).  The nice thing about the StackExchange is that you can read multiple comments with different perspectives.  This article points out that using the definite article before a proper name is a southern German-speakers convention.  It's a pattern that's clearly used in places like Bavaria, and down south into Switzerland and Austria.  


4.  Speaking of understanding German... One Saturday evening I was attending a concert at one of the local churches, which are often venue for chamber groups.  The director stood up at the beginning of the concert and started speaking in German.  "No problem," I thought, "I can understand this!"  I listened happily for a couple of minutes until suddenly, everything changed:  He was still speaking, and it sounded like German, but I couldn't understand anything!  Can you explain what happened in my few minutes of non-understanding?  

For this Challenge, I started simply with the query: 

     [ language in Switzerland sounds like German ] 

where I quickly learned about Swiss German.  I guess I'd learned about it before, but I'd never noticed it in a public forum, and read several articles--here's one from Quora--about how it really IS different than standard "High German" (Hochdeutsch).  Here's another article about the differences, but the differences are substantial, which explains why it would be unintelligible to a German-speaking neophyte like me.  

Search Lessons


1. Reading in a topic area after a simple query can often answer questions.  Although my initial query about the milk dipper didn't initially answer my question, after skimming through images to find near look-alikes, it became clear what was going on.  

2.  When searching for a difficult topic, consider the possibility that you don't have the right query terms!  In particular, remember to think about using an alternative way to say something.  Using a reverse dictionary can often give you insights into other terms.  As you saw in the above example, you can suss out other works that you can't think of by trying out phrases you think would be in a plausible definition. (That's what I did with the phrase [ "cup used to"] on OneLook's reverse dictionary.)   
Search skills don't make cultural differences vanish, but they definitely make the world more intelligible!  

Search on! 


Thursday, May 10, 2018

Search Research Challenge (5/10/18): Swiss mysteries?


I've spent a fair bit of time in Switzerland... 

... over the past couple of years.  Partly it's because I go there for work, but part of my work last year was teaching a semester-long class in Social Computing at the University of Zürich.  

But as with any place where you're the new guy, there are certain mysteries that crop.  Here are a few that came up for me.  I figured them out, but can you figure out as these little trans-cultural mysteries as well?  (Of course, if you're Swiss, you'll have a huge advantage here..)  


1.  When in Switzerland, one naturally eats a lot of chocolate.  (I certainly did.)  But as I was munching on a bar of Lindt milk chocolate, I realized that the image on the bar of chocolate made no sense to me.  WHAT is going on with this cup?  Why is the handle so strange?  (Did Dali do this illustration for them, or what?)  




2.  One day I walked into a bakery in central Zürich and spying this delectable pastry, I ordered a croissant.  I was very surprised when the clerk said, "I'm sorry, we don't have any croissants."  But this is what I saw in the display case:  



Obviously, I successfully ordered this item. What SHOULD I have called this thing that looks-like-a-croissant?  When in Switzerland, what are these called?  
  

3.  I understand more German than I speak, but every so often I would hear someone say something odd.  One construct I heard that seemed odd always involved people's names.  For instance, "Wo ist der Hans?"  Can you figure out why that sounds odd to me (a native English speaker), and why the word "der" is in that sentence? 


4.  Speaking of understanding German... One Saturday evening I was attending a concert at one of the local churches, which are often venue for chamber groups.  The director stood up at the beginning of the concert and started speaking in German.  "No problem," I thought, "I can understand this!"  I listened happily for a couple of minutes until suddenly, everything changed:  He was still speaking, and it sounded like German, but I couldn't understand anything!  Can you explain what happened in my few minutes of non-understanding?  



I suspect that Regular SRS readers will make short work of these questions.  But knowing how to search for these small cultural questions as they arise was a great source of comfort to me as I was navigating throughout central Europe.  Search skills don't make cultural differences vanish, but they definitely make the world more intelligible!  

Let us know how you found the answers!  

Viel Glück!  

Search on...