Wednesday Search Challenge (May 25, 2011): Make a great question, win a great t-shirt!

Okay, so technically speaking, today's Search Challenge isn't a challenge of the kind we've had traditionally.  Nope.  Today's challenge is for real stakes!  


As you know, the AGoogleADay.com daily search puzzle has been running since April 11th.  We're really happy with the way it's going, but it's a bit of a black hole, consuming lots of time and energy as we race to write questions well ahead of publication date.  


So I'm hoping you can help out.  


The challenge is simple:  If you write a question that we can use on AGoogleADay, we'll send you one of our very rare, very limited edition AGoogleADay t-shirts!  


You've already seen the AGAD puzzles, so you know how they work.  Short, easily understood problem; an answer that's easily verifiable and (to fit into the master plan!) teaches something about how to search along the way.  


Turns out that writing these questions/answers isn't exactly easy.  While it's easy to write really trivial (but uninteresting) questions, it's harder to make questions that are verifiably correct via a search engine AND have an answer that's succinct and to-the-point.  The puzzler needs to be able to recognize a correct answer when they see it, and our automatic "correct answer" code needs to be able to determine that what someone enters is in fact the same as the right answer.


Thus, questions like:  "Give three causes of the Spanish-American war" aren't great--both because it's difficult to actually tell if you've got the right answer, AND it's hard to know if what you typed into the tiny answer box actually IS a correct answer.  


See what I mean?  


If you have a question and answer you think will work, please send it to me at drussell+AGAD@ work.  (The +AGAD will let me filter your answers into a labeled file so I won't get overwhelmed with emails.)  


Make sense?  If so, send me your questions (and answers--while I love to search, I can't spend ALL my time checking your answers).  


Anyone sending homework problems will be severely reprimanded!  


Start writing!  


On Friday I'll comment on some of the problems I've received, and talk about how to improve the questions.  


Good luck! 

Answer: How can you find a book by its color?

Whoa!  How did it get to be Tuesday of the next week?  


Sorry about the delay in getting the answer to you.  Between the AGoogleADay.com activity and a few other things going on, it's been a bit since I've been able to get back here to till the fields of SearchResearch.  


So, a quick answer to the Challenge from last week:  A kid walks up to you and says:  "I'm looking for a book about Rosa Parks, but I don't remember the title.  All I DO remember is that it's got a picture of her on the cover... and the cover is mostly green."


As I found out from friends, this specific problem  is actually fairly easy to do on regular Google Book search.  (Just search for books about "Rosa Parks" and you'll see the covers.  As it turns out, the green book is in the first set of books.  Unlucky for me, lucky for you!) 


What I meant to point out was the trick of using Google Image search to find books by various image attributes.  That is, you can go to Image search, search for [ Rosa Parks book ] then filter by color (or face, or lineart, or whatever option is there).    

Several people wrote to say they'd found the answer quickly.   Nice. 


Search lesson:  Sometimes doing a search for something in Images is a great idea... especially if it's a visual property you're trying to search on (such as "green book cover").  



Okay... tomorrow... a more difficult challenge!  One worthy of all you uber-searchers out there!  


Search on!

Wednesday Search Challenge (May 18, 2011): How can you find a book by its color?

Again with the running a day late!  

Sorry about that... I ended up doing 32 press briefings about AGoogleADay.com between 4AM and 9AM on Tuesday morning, and that had the side-effect of giving my regular Wednesday schedule an entirely new look and feel!  


Today's Search Challenge isn't that hard--once you figure out the trick.  It's pretty simple if you know.  


A kid walks up to you and says:  "I'm looking for a book about Rosa Parks, but I don't remember the title.  All I DO remember is that it's got a picture of her on the cover... and the cover is mostly green." 


Can you find the book?  Who's the author?  


0 - 1 minute:  Über-searcher 
1 - 2 minutes:  Power-searcher 
2 - 5 minutes:  Need to learn a few new tricks!
> 5 minutes:  Read SearchResearch more often!!!! 




Search on! 

Synthesizing knowledge with Google Spreadsheets

Sorry about the delay in answering.  I spent last week at the Computer-Human Interaction conference in Vancouver, hanging out with my user-experience colleagues and enjoying Vancouver's flowers and waterfront! 


As you recall from last week, my friend asked:  "Can you tell which of the Fortune 200 companies have been around for more than 100 years?"


It's a great question, but not an easy one.  Here's how I solved it.  I followed much the same process as Fred Deventhal (see his comments from last Wednesday).   Nice job, Fred! 


Another regular reader, Ross Nelson, tried to solve this using Google Squared. That approach almost worked, but fell apart because you don't have quite enough control over how it works. 


My solution... 


After poking around for a while with queries like [ Fortune 500 start date ] and [ Fortune 500 1700..1910 ] I realized that the start date of a company is typically called its "founding" -- as in, AT&T was founded on ....  


Tried that for a while, and got nowhere either.  So. HOW can I get the founding dates?


I knew that for any ONE company I could easily look it up (say, on Wikipedia, or the corporate web site).  But doing that repeatedly would quickly grow old.  How about if I start with a list that's easy to get--just the names of the Fortune 500, from which I'd pull off the top 200.


Changed my query to [ list of  Fortune 500 ]   that worked well and go me to this:


http://money.cnn.com/magazines/fortune/fortune500/2010/full_list/101_200.html


Now I've got the list.  I copied/pasted the top 200 company names into a Google Spreadsheet ("Fortune 200 Spreadsheet")  and then wrote a simple GoogleLookup function to take the company name and lookup (using Google) the "founded" date.  


If you look at column C (the "Date Founded") column, you'll see each formula looks like this: 


=googlelookup(B2, "founded")


This is the Google Spreadsheet Lookup function that searches for the "Founded" property on each of the company names from column B.


As you can see, it actually worked pretty well. I had to go a fix up only a couple of the entries (which returned the date plus a bit of extra text). The fixed-up dates are in column D ("Int version" -- where "Int" stands for "integer").
A little calculation later, and you get the years-as-a-company column (E).


If you look at the right side of the spreadsheet, you'll find two charts I made for fun. The first one "Years as a company") shows the distribution of companies by age. The downward trending slope is what you'd expect if there were companies founded roughly every year that made it to the Fortune 200. So that's not a surprise.

I then made a version with companies grouped by decade using the Spreadsheets 

histogram function.

Search lesson: When you've got a search task that requires looking up a lot of very similar pieces of data, consider using a tool to help out. This one worked pretty well. In future posts I'll talk about more complex ways to do similar kinds of searches over larger, more diverse data sets.


Search on!

Wednesday Search Challenge (May 9, 2011): Which companies are more than 100 years old?

My friend Alison asked a simple question:

"Can you tell which of the Fortune 200 companies have been around for more than 100 years?"


You wouldn't think it'd be that hard.  But I found it using an interesting method that you might not know about.  


Clearly, you have to figure out what the Fortune 200 are.  That's not too difficult—it's just the current Fortune 500 list ordered by revenue—take 200 off the top and you've got it.  


Once you've got that list, just figure out when all 200 companies were started, and then you'll know.  


Here's the trick:  you probably don't want to do this by hand, one company at a time.  So... how can you search for all 200... at once? 


Tomorrow's answer will reveal all!  Note:  It took me about 20 minutes to solve this.  


Search on!  

Getting the answer right

As you can imagine, finding answers is easy--but showing that they're right is another thing.


In the AGoogleAday.com daily puzzle, we're constantly learning what makes for a good puzzle (short, easy to state, clear method for solution, unambiguous answer).  And it's that last point that makes life in a Google-world much more difficult for puzzle-writers.


Last week, for example, we gave the following puzzle:

This is a nice puzzle.  It's short, there's a pretty clear method to solve it... but it's a little ambiguous.


What we meant to say was:  "You are standing in the farthest west incorporated U.S. town in the lower 48 states with a population of one person.  What is the official, posted speed limit?"


But, as you can see, it's a longer, less appealing question.  Yes, there are towns in Alaska and Hawai'i that have 1 resident, but in our defense, they're all unincorporated (at least the ones I found).


It's not a great question, though, because--who knows?--since we wrote this question, a town might have turned into a population-size-one town, and the answer wouldn't reflect the current state of affairs.  Rats.


So it's a bit tricky to write these kinds of questions.  Especially when there are multiple potential ways of answering the same question.  For instance, many puzzlers wrote in to point out that Quirky Travel Guy said the speed limit was 40 mph.  We report the speed limit as 30.  What's up with this?  


The big, fundamental point I would love to make in the puzzle is that while people can FIND answers, they need to also learn to evaluate their level of credibility.  A self-admitted "quirky travel guy" (while very entertaining), is not nearly as compelling as seeing the speed limit sign at the edge of town via StreetView.  (Use the plus-widget in the map streetview panel below to zoom in on the speed limit sign.)  



View Larger Map


And that's the drawback of the short-form of the puzzle.  I'd love to be able to write longer back-stories like this, but can't quite fit it into the 420 character limit of the answer space!  


Search lesson:  Always second-source your findings, and when in doubt, choose the one that seems more likely to be true in the long run.  


Lesson for teachers:  There are always interesting other interpretations of your questions.  Think about those when designing your test questions!  (And, most importantly, try out your test questions before assigning them to students.  Writing good questions is an art!)  

Answer: How common are animal-adjectives?

There are many ways to answer this question, and in his comment, Hans goes from [ frequency lists ] to Wiktionary to the Project Gutenberg word count lists.  


From the search point-of-view, that's not a bad search, but here's what I did for contrast.  


I started by framing the problem in the most obvious way I could think of with the query: 
[ word frequency over time ] 


Yes, I know the term "frequency" has a notion of time built into it, but I was searching for a web page that would write over the changes "over time" in relative term use.  That's why I included "over time" in the query.  In essence, this is a two-part query:  "word frequency" and the idea of "change over time."  I left out "change" because I thought that would mess up the bigram whereas "over time" would be frequent enough to be useful.  


As I sifted through the results, I noticed something I should have thought about straight-away:  the Google NGRAM Viewer project!  This is a marvelous "digital humanities" tool that would give me exactly what I wanted in one convenient package!  


In this case, once I'd found the right tool, the problem becomes trivial.  I did a comparison query of  piscine, lupine, vulpine, and bovine to generate the following chart.

Why so common?  Clicking through on the links at the bottom of the page and comparing hits from the 1900s vs. the 1980s shows why: Agriculture and disease.  The first bump in the 1910s is agricultural.. but then "bovine spongiform encephalopathy" starts the rise to bovine's 
current frequency levels.  

In this case my suspicion about the relative occurrences of the terms is confirmed.  Bovine really IS much more frequent than the other terms.  It's SO frequent that I had to remove it from the query to see more of the details on the other terms.

Still... lupine is rare.  But why does it dominate the other two "animal adjectives"?  My best guess from looking at the books in the hit list is that this the period when lupine (the plant) began to be taken as a serious topic of horticulture.  


(Among the other things I discovered while writing this post was that "Current Minnesota recommendations are that white lupines are unacceptable for growing pigs (under 225 lbs). A 1988 Minnesota study reported a 2% reduction in feed intake for each 1% lupine in the diet..."  Alternative Field Crops Manual, "Lupine," Nov. 1997.  Or, in other words, "lupines not suitable for porcines.")  




So the search lesson is clear--when you find a tool that does what you want, use it.  And in fact, search for the tool first.  You could save yourself a large amount of time! 


Search on!

Wednesday Search Challenge (May 4, 2011): How common are adjectives describing animal-like characteristics?

In yesterday's post I talked about writing puzzles.  But the Wednesday search challenges here are somewhat different--they're (mostly) real search problems that people have had  Usually I'll be talking with someone and they'll say "gee... I could never find out about foo... " and away we go into another challenge.  


As you know, I love words.  Recently a friend and I got into a discussion about how common various kinds of adjectives are.  The easy ones are easy to estimate:  red is common, angry is common.  But what about others?  How common is something like florid?  


When word people talk, they often end up talking about extreme cases--and this conversation went that same way.  "What about animal-adjectives like piscine, lupine, vulpine, or bovine?"  


You might think this is just so much how-many-angels-can-dance-on-a-pinhead kind of discussion... but no... gauging relative term frequency (the fancy way to say how common a word is) is a very useful bit of knowledge when constructing search queries.  If a term, say vulpine is very common, then you'd wonder if it means something other than what you might think.  You might wonder if there's a common usage that's unknown to you.  


So this all leads to today's search challenge:  What is the relative frequency of the animal-adjectives piscine, lupine, vulpine and bovine? 


And... because we're word people here, how has the relative frequency changed over the past 100 years?  For instance, was the word "vulpine" much more common in the past?  (It's relatively rare now.)  


To spare you the lookups: 


piscine... like a fish
lupine... like a wolf 
vulpine... like a fox 
bovine... like a cow 


Any ideas? 


Search on! 

Making up search problems -- more complex than you might think

The AGoogleAday.com puzzles aren't easy to write.  I'd wager they take longer to write than to solve.  The problem is getting the right balance of interesting topic, having a clear solution and actually teaching something interesting to the puzzler.  You see, the goal of AGAD really is to illustrate the way search works, largely by example.  


For instance, today's puzzle: 

is actually fairly simple.  It's on an interesting subject area and illustrates an important search concept, to wit:  choose your search terms carefully.  


Search beginners make beginner mistakes, often including too much in their query.  A query like [ hottest cousins ] or [ hottest cousins rank 10000 ] are doomed to be unsuccessful--they're too generic (or, in this case, probably NSFW unless you have safe search turned all the way up!).  


But a wise searcher will recognize that the term "SHUs" is an odd thing.  You might not recognize it (in which case a quick [ define:shu ] will yield an appropriate definition), so it's probably material to the investigation!  


I solved this with a simple [ 10000 SHUs ], and scanned the snippets.  It's only Tuesday, how hard do you want it to be?? 


But if you're a teacher making up search problems like this, figuring out how to write the problem can be tricky. 

I generally start on a topic and then explore around for interesting side-bits of information.  I actually keep a notebook in my pocket and write down good ideas as I run across them in the real-world.  I see a blimp passing overhead and start to wonder... what do I not know about blimps... a few queries usually reveal a world of information that can be converted into a puzzle question for teaching search skills.  


A common mistake that puzzle-writers make is to pick on some piece of information that's SO obscure that nobody will know it.  (Example:  "what's the 1,000th decimal digit of pi?")  Nobody knows it because it's effectively random.  Boring.  


But if you start to wonder about blimps--what gas IS used in blimps?--you'll find out something interesting.  If you then start to find out something interesting about helium... then you've got two parts to an interesting puzzle.  I'd link those together into a puzzle something like this:  "There have been two gases used to float blimps.  We're not about to run out of one kind of gas that was used to lift blimps into the sky, but we ARE going to run out of the other gas.  What is that naturally occurring gas that we're in danger of exhausting?"  


It's a naturally interesting question, and teaches you something along the way... the way all great teachers do.  


Search on!   And teach along the way! 

Trick of the day: copypastecharacter.com to simplify inserting special characters

When you write or edit, you often need to insert a special character into your text.  How do you do it?  If you're like me, it's probably by dropping into a special insert symbol sub-mode, which is fine, but sometimes laborious.  In MS Word, for example, you can use their Insert>Symbol menu choice, but then you're presented with ALL of characters.  Yet, most often you need to insert just curly-quotes, plus-or-minus, the Yen symbol, or equivalent.  


I found a web site that's marvelous:  www.copypastecharacter.com -- it does just what it says.  When you click on a symbol, it puts it into your keyboard copy-buffer.  That means you can now go back to your editor (say, Gmail message window) and Paste (Control-V) that character.  VERY simple, VERY nice.  

I usually keep it in a tab somewhere handy.  (FWIW, I also shrink it slightly to make sure it completely fits onto my screen... a single Control- shrinks it just a bit.)  


Also note that if you hold down the ALT key, you can click on multiple characters to get a few chars all at once.  (Example:  ≠~÷≈∞ )  Handy when you know you're going to use all those characters in a single message.  


And for people who write HTML, note that you can get the HTML equivalents for any of the characters!  Click on the "As HTML" button at the top of the page, and you can generate things like & iquest; (for the Spanish 'inverted question mark' symbol, ¿).  

You know and I know that search engines don't work so well to find special characters.  Generally, special characters are dropped from the search string.  The exceptions are special characters used as operators (e.g., [ salsa -dancing ] or [ +joiker music ] ), and the characters hash (#), ++ (to handle C++).

Write on!