SearchReSearch

Wednesday search challenge (Feb 1, 2011): Where are you?

2012-02-01T06:15:00.000-08:00

Imagine the following situation: You've had a sudden episode of amnesia while on a trip and have suddenly just woken up in a strange place. You have no idea where you are. All you know is that this is the view out the window.

You need to get your anti-amnesia medicine to prevent blacking out again and you need it fast.

Insurance will cover the cost of sending your medicine via courier, but you need to know where to tell them to go.

Question: Where are you? Can you find the street address, in what building, in what city?

For today's extra credit--can you figure out what office number and floor to send the medicine? How about the phone number?

Search on!

And remember... be sure to tell us HOW you figured it out and how long it took you to do! (One of the points of this blog is to share the search methods and tricks we know!)

(And a tip of the hat to Andrew Sullivan for his inspirational "The View From Your Window" series.)

The fourth R--Research, and the skills we all need

2012-01-27T07:08:00.000-08:00

We all know about the three R’s of education—reading, writingand ‘rithmetic. The three basic skillsthat school have to teach… and which obviously doesn’t include spelling.

I want to propose that there’s a 4^th R we shouldbe considering: RESEARCH.

If you think about it, learning has changed from aschool-only activity to a life-long activity. And just as advantage accrues to the person who can learn the best andknow the most, so also does the ability to research to the best of yourability.

As SamuelJohnson said: "Knowledge is of two kinds, we know a subjectourselves, or we know where we can find information upon it."

While that’strue, but this common version of his quote usually leaves off the rest of thatparagraph: “...When we enquire into anysubject, the first thing we have to do is to know what books have treated ofit. This leads us to look at catalogues, and at the backs of books in libraries.” (Boswell's Life of Johnson, 1791)

Inother words, even if you know how to research something, you still need to knowa little bit about the skill of how to search. In Johnson’s day that meant knowing that cataloguesexisted, that libraries were collections of books on topic of interest, andthat the back of a book contains an index. It also meant that you knew how toget into a library, many of which were still private and by subscription (read,“invitation”) only.

People fluent in search and retrieval not only savetime, but are far more likely to findhigher quality, more credible, more useful content. More importantly, they can ask questions thatwere impossible just a few years ago. People with these skills are effectively smarter.

Using Google to do search is easy. It's been designed that way. You type something like [New York Times] intoa search box and a moment later you're reading the paper. If you search for [pizza Mountain View], you get a list of local pizza places withphone numbers and user reviews.

Most of the searches that Google sees in a typical day fallinto this simple category where user goal is clear and the results are pretty obviousand unambiguous.

But a significant number of searches are not. Searchers might have a goal in mind but theycan’t figure out how to express it in a way that will give them what they want. Sometimes their search is precise, but theydon’t know how to read and interpret the results. Sometimes I’ll see searchers spending 30minutes searching for something that should take less than 2 minutes. It drivesme crazy as a researcher because I know that the searcher is missing just one small,but critical piece of information. Wetry to build as much as we can into the search algorithm, but people still needto know a bit about how the web is organized (there’s no index in the back ofthe book) and how search engines crawl, index and respond to their queries.

In a sense, that’s my mission—to help people become betterresearchers, beyond just the basic skill of knowing how to make Googledance. My goal is to help people understandthe larger issues at play here—how to be a literate person now, and now to becontinually learning how to be literate as changes happen in the future. This is the idea of meta-literacy—knowing how to be literate about your ownliteracy. More about this in future posts.

BOTTOMLINE: Research is a skill that we all take for granted, yet it’s acritical skill for our future. As thenature of work and education changes (and that, really, is the only constant wehave), we… as a teaching culture… need to bring our students up to speed onwhat it takes to be good searchers.

We need to give them the skills of the 4^th R—research—and all of the skills andknowledge they need to function effectively as learned searchers.

What’s more, we’re trying to equip them with skills they canuse not just now, but for every information search problem they confront nowand in the future.

Search on!

Answer: What was Arthur Thorpe's job?

2012-01-26T11:09:00.000-08:00

Here’s the quick answer: Arthur A. Thorpe was an osysterman living at 1603 California St in SanFrancisco, 1899. He was reknown not justfor his mustache, but also for his skill at bowling (10 pins). He’s mentioned several times in magazines andnewspapers of the time for having bowled a perfect game (that is, with a scoreof 300—a much tougher and rarer thing in 1899 than today).

Now, the backstory…

I was thinking about how hard (or easy) it would be to learnabout the life of a person living in San Francisco at the turn of the 19^thcentury. I stumbled across a telephone bookof SF for 1899 and started reading. Weird, right? But let me tellyou, the directories of 1899 are VERY different than what you might think.

As you can see, people were pretty open about sharing all kinds of information about themselves. (And you thought this was purely a Facebook phenomenon!) As you can see, not only are some really interesting occupations listed, but if you were unemployed, your social position was listed. Some of the more interesting jobs I noticed: “cigar box maker” “fancy card writer” “engrosser of resolutions” “Dow steam pump operator”… but then again, you get the occasional “widow” or “unemployed” along with the usual bakers, machinists, pressmen and telephone operators.

In my search path, the key was to realize that (A) I needed to find some master listing of people and their jobs, and (B) directories in the late 1800’s would list the person’s job title along with their street address.

So my first query was for [San Francisco directory 1899] My hope was to find a master directory of everyone living in SF in 1899, and that's why I included the term "directory" in the query--that was a description of the kind of thing I needed.

This led me pretty quickly to the Crocker-Langley directoryfor the year 1899. and one click there to the full-text of the document. A quick Control-F in that text document (once it finished loading… it’s pretty long) showed me that Arthur A Thorpe was an oysterman working for the Darbee and Immel Oyster Company. (Note that Arthur Thorpe was the 41st in the list.)

As good as this is, I really wanted to see the original scanned version. A bit more searching for the Crocker-Langley directory [Crocker-Langley directory 1899 filetype:pdf] lead me to the PDF of the scannedversion. (I took a bit of a chance by using the filetype:pdf restriction. I was just guessing that someone would havescanned it. I was right.) It’s much more satisfying to read than thesomewhat errorful OCRed text-only version.

The page with Thorpe’s entry is shown along with an ad for “Kellogg’s Whisky: Cure la grippe”… somethings never change.

How many oystermen were there at the time? If you just do a Control-F in Chrome on thetext file, you’ll see there are 48 hits for the word “oysterman” – so therewere at least 48 such people listed in the directory. (However, I also noticed that while there arepeople listed as “Chinatown Tour Guides” for their employment, there are very few Chinese names in the directory…so I suspect that more than a few people didn’t make it into the book.)

I wanted to estimate the total number of oystermen on thebay living in San Francisco at the time, so I then did a quick search for [ “oysterman”OR “oystermen” OR “oyster man” “San Francisco” 1900 ] hoping to find a resourcethat would discuss their lives. Sureenough, I found Bulletin 123 from the California Department of Fish and Game(1963)

A compelling text about the history of the oyster industry. (I’m not kidding—it’s really interesting toread how the Dept of Fish and Game hoped to exterminate bat stingrays in theSan Francisco bay ecosystem! It’s a lookback to a far simpler view of fisheries management.)

But within that text you’ll find that San Francisco bayproduced around 2.7 million pounds of oyster meat. Now you have to take into account that therewere oyster beds all around the bay (including some very close to the Dumbarton landing, not far from where theGoogleplex is located today!), so just 48 oystermen seems a bit small.

It’s difficult to get an exact count, but a bit of searchinto the 1900 census (available at http://www.census.gov/prod/www/abs/decennial/1900.html)shows that in 1900, California as a whole had 1,969 people listed as “fishermenand oystermen.” Other document suggestthat San Francisco was about half of all California fisheries output. So agood ballpark estimate of oystermen at work in San Francisco is somewherebetween 48 and 123 (my back-of-the-envelope calculation: 1969/2 = 985, the number of SF workers. Guessingthat oysters are 25% of workers in the fishery in 1899, is 985/4 = 246. If oystermen in SF were around 50% of all theoystermen in the greater Bay area, then 246/2 = 123).

This is very much in line with the Dept. of Fish and Gamereport mentioned above (from 1963). Intheir section on oystering in San Francisco bay between 1875 and 1910, theywrite: “Reports on the San Francisco Bay oyster industry state that about 100men worked on the beds (Hittell, 1882, p. 364; Townsend, 1893, p. 358; Wilcox,1895, p. 207). Townsend says that this number was increased considerably duringpeak activity, probably during harvesting for the winter holiday season andduring seed planting in the early spring and fall. About 90 percent of the menwere single, itinerant workers of Scandinavian origin, mainly recruited fromthe San Francisco waterfront.”

That suggests that not all of the oystermen would have beenin the directory. So our estimate of 48 –123 oystermen in San Francisco in 1899 is pretty reasonable.

Now, to find out more about Art Thorpe’s life, his hobbiesand such, I turned to Google Books with the search [ “Arthur Thorpe” SanFrancisco 1899..1905 ] (I used the date range operator to give me hits aroundthe 1899 time and to exclude hits from other Art Thorpes later.)

That took me to the magazine “Western Field: The sportsmans’magazine of the West” (vol 3) and there I found that he was a crack bowler.There I found both his picture and the following text:

He’s mentioned throughout several more publications as amaster bowler, even gaining a mention in Hawaiian newspapers for his feat of bowling a perfect game and a bit of notoriety for his high stakesbowling games in 1904. (Link to SanFrancisco Call newspaper archive for 1904)

Search lessons: There are many to take away from this particular challenge. But let me highlight just one for today: When you're looking for a particular kind of thing, you need to include the term that best describes it. In this case, we used the word "directory" to get to the San Francisco directory from 1899. Without using that word, it's really tough to find your way to another collection that will work nearly as well.

So, when you get stuck on locating a particular kind of document, consider adding in a descriptive term (e.g., "directory" or "yearbook" or "album") as a way to provide additional context to the search. With it, you'll likely be much more successful!

Search on!

Wednesday Search Challenge (Jan 25, 2012): What was Art Thorpe's job?

2012-01-25T05:46:00.000-08:00

Forgive me if I ask you another question about historic San Francisco. But I ran across this intriguing man in the mists of San Francisco history, and I can't pass up the chance to teach you another great search trick in the process.

Meet Arthur A. Thorpe. He lived in San Francisco at the turn of the century... the one before the last one. To make things easier for you, I'll even tell you he lived in SF in 1899.

We know he enjoyed several hobbies (I'll let you figure out one of them), had a pretty nice handlebar mustache, centrally parted hair, and had an occupation that was once common on the Bay, but is VERY rare these days.

Can you figure out what Art's job was during 1899?

And, for extra credit, can you estimate how many other people in San Francisco had that job as well?

When you post your answer, please include the URL where you found the answer, the search path you used to figure it out, and how long it took you to get to the secret!

I'm guessing that this is difficulty level 3. (This is on my informal Russellian scale where the max difficulty is 5. Level 1 is a simple 1-query search that takes less than 30 seconds, while a 5 is a full day's worth of work with many queries along the way.)

Search on!

Search strategies and tactics

2012-01-24T13:42:00.000-08:00

First, let’s get some definitions in place: An information goal is yourobjective—what you want to find out. Itcan be very specific or very broad; simple (“what time is sunrise today?”) orcomplex (“what’s the best internet service provider for my startup inBangalore?”).

And now, let’s talk about what a strategy really is…

- a searchstrategy is the high-level planyou create and

follow to achieve your objective

- a searchtactic is what you actually do to follow the strategic plan

Obviously, there’s a lot of back and forth between these; the dividing line isn’t bright and clear, but more of a slippery slope. Yourgoal might shift based on what you learn. Your tactics might vary as you find roadblocks or problems in yourway. Those little problems that come upmight cause you to re-think your strategy—making you pull up for a minute and think ofanother way to approach the problem. (One way to approach the problem is to re-define your goal…)

For instance… Assume your goal is to find out what the roleof mercury was in gold mining during the California 1849 Gold Rush. Your goal is to find out what is was usedfor, how much was used, and what happened to it after it was used?

Sample Search Strategy:

Your search strategy might be to find some secondary sources(Wikipedia, histories of California) and get a quick overview.

Then, you might drill down into primarysources to see what they said at the time. (You might check newspapers sites for archival information about theGold Rush, locating a bunch of resources.)

You’re probably going to download them, then process carefully, lookingfor leads to other companies and people that were involved in importing,selling or using mercury.

From that set of leads, you can expand your search toinclude specific archival searches for information about quicksilver companiesin northern California, etc etc etc.

That’s one thing about a strategy—you start with a more-or-lessvague strategic plan that gets turned into tactical action over time. In other words, your search tactics thenimplement your strategy.

So… getting back to the point of this post, I originallyasked “What search strategies do you use?” And I got back a bunch of answers; many of which I would considertactics. Let’s look at what was posted… (I’ve re-written much of what people actuallyposted for clarity.)

Tactics:

1. Scoping – limitingsearch to a particular kind/place/genre (example: using site: to limit)

2. Use left-side of Google search page to do specificfilters (e.g., translated pages, image search using color)

3. Use Google Videoto search ALL of video (rather than just YouTube)

4. Use another search engine when stuck. Be aware of other search engine capabilities(e.g., Alpha) and limits (IMDB, Twitter)

5. Know how to use synonyms in search.

6. Use Image search to find definition of unfamiliarterms (e.g., [ polynya ] or [ planetarygear ] )

7. Imagine the way in which the answer to your questionmight be presented (chart? Infographic? Visualization? List? Album?). Use those terms as “context terms” in yoursearch.

Strategies/Strategicknowledge:

1. When being literal isn’t working, try a more descriptive search (thatis, use “context terms” to shift your search) (example: searching for [ free books on kindle fire ] while on Amazon)

2. Know when to ask someone else for help. (And have a developed social network of expertsand people smarter than you.)

3. Recognize when you’re getting stuck in your search (e.g.,when you’re getting the same results over and over).

4. Speak aloud to articulate what you’re really looking for(aka, the Teddy Bear strategy)

5. Walk away and come back later.

6. Recognize that languagechanges over time and that searches for information in history might use verydifferent terms for the same idea. (Example: use Abyssinia instead of Ethiopiawhen searching pre-1920)

7. Vocabulary is important – if you’re having a tough time,then you’re probably going to have to expand your vocabulary! (e.g., while looking to mount a chin-up bar,needed to find a widget to attach the pipe…. Figure out that’s call a “pipeflange” – go from there)

8. Learn something new: For newsearch/research topics, plan on learning something along the way… new terms,new concepts.

9. Don’t give upafter first 3 tries

10. Try different combinations of search terms

11. Be willing to recognize that you might be in the wrongplace

12. Backtrack through references – If someone’s name keepspopping up, search for their original work

13. Identify the most unique searchable terms. Look for"individualistic" terms (that is, low frequency terms that areclearly on-topic).

14. ALWAYS plan to double-check to find a second (or third)source for important facts

15. Start broad, learn a bit, then refine your search onceyou know the direction you want to go.

16. Start with an overview article on the topic. (Wikipedia is great for this.). Follow links outward from there to otherplaces you’ll want to go.

In posts ahead, we’ll dive into the details of thesestrategies (and maybe come up with few more)!

Answer: Which steeple is oldest?

2012-01-19T07:46:00.000-08:00

I knew this was a bit harder, but a good number of readers managed to solve it without too much difficulty (although the average time-to-solution went up significantly).

For me, the key to this was trying to figure out each of thesteeples by using Image search. (I just saved each image to my computer, thenuploaded them into Google’s Search By Image. Link for a description of how to do Search by Image.)

Of the four steeples, only one worked with Search by Image—that was for steeple#3. From that match, I knew it was St.Michaelis Church in Hamburg, Germany, dating from 1786. But what about the other steeples? I knew nothing about them whatsoever andImage search wasn’t working.

So I tried testing a reasonable assumption about the image collection. While they could be steeples from all over the world, I was hoping that theyall might be near each other, maybe even in the same sity. Since they all looked to be Germanic inconstruction, I tried doing regular Google Image Search with [Hamburg steeple]and started scanning the pictures.

Sure enough, there they all were—slightly different views,different times of day, but all recognizably the same.

Search lesson: When you’re stuck, making a key assumption toreduce the size of the search space is often a good idea. Be careful though—you don’t want to get stuckin the rathole of an assumption you made that isn’t panning out.

Luckily, in this case, it worked out perfectly. I was able to easily match the photos to the images in the search results:

#1 as St. Catherine’s Church, #2 as The Rathaus,

#3 as St. Michaelis, and#4 as St. Nikolai.

And now for the next question—which is the oldest?

While they’re all fairly old, they’ve also all undergone significantreconstruction over the years. So theproblem turns into a bit of an “Abraham Lincoln’s Axe” problem. (You know, if you have Lincoln’s axe, thenreplace the handle, and later replace the head, do you still really haveLincoln’s Axe?)

For churches like this, especially ones in Hamburg (whichwas largely leveled during Operation Gomorrah), the “date of construction” turns into aquestion of “when was it started” and “when was it reconstructed” and, just asimportantly, “when was the steeple built onto the body of the church?” Sometimes the steeples were added muchlater.

Doing the obvious searches for each church, you can find thefollowing:

#1: St. Catherine. Thechurch body has parts dating back to 1256, but the steeple seems to date to1657. The church was restored between1950 and 1957.

#2: Rathaus. Thisversion of the building was started in 1886 and opened in 1897. It too was destroyed in the air raids andrebuilt afterwards (I assume during the 1950s, but wasn’t able to confirm thisdate).

#3: St. Michaelis. This is the 4^th or 5^th church on this site(depending on how you count), but the current architectural version was builtin 1786, then rebuilt after a fire in 1906 (be careful with that soldering!), andrebuilt again after the war in 1952.

#4: St. Nikolai. (Or St. Nicholas, in English) Started as a chapel that opened in 1195,that church burned down in the great fire of 1842, then rebuilt in a neo-Gothicstyle with a steeple that reached 486 feet. Also destroyed in the war by Operation Gomorrah, it was never rebuilt,but left as a semi-ruined hulk as a reminder of the devastation of war.

So.. which steeple isoldest?

If you go by “original building date,” it would have to beSt. Nikolai. If you go by “date of laststeeple reconstruction,” that would also have to be St. Nikolai, as the steeplewas completed in 1874 and then never reconstructed after the war (althoughstructural work has been done to make sure it doesn’t collapse).

The truth of this particular story is that I took thesephotos while I was in Hamburg late in 2011. I was steeple-hunting (a hobby a bit like train-spotting) when I saw St.Nikolai from about 1 mile away. I couldn’tfigure out what it was, and when I finally made it to the site, I was struck bythe place—beautiful, horrible, sacred and profaned. It’s a moving testament to peace in a lovelycity.

Search on!

Wednesday search challenge: Which steeple is the oldest?

2012-01-18T04:41:00.000-08:00

Today, a slightly harder puzzle for you to solve.

Below are four pictures of steeples that I've encountered over the years. I'm quite taken with steeples, finding them fascinating, and love to take photos of them as I travel around. They're tough to build, full of symbolic meaning, and seem to always have a good story associated with them.

But today I have a simple question--Which of these is the oldest steeple? Can you identify each steeple and give its construction date?

How hard can that be?

When you leave your answer, be sure to tell us how long it took you to find the answer! (And if you could NOT find the answer, that would be useful to know as well.)

Search on!

Google Alerts--standing queries to monitor the world

2012-01-17T12:29:00.000-08:00

I don't know about you, but I really don't have as much time as I'd like to scan all of the new journals, web publication sites and News feeds. Instead, I rely on Google Alerts to keep me up-to-date and in-the-know.

Google Alerts are basically "standing queries." You write a Google query, then decide how often you want it to run and over what body of content (news, web sites, etc.).

Since it's a regular Google query, you can use any of the normal advanced query operators. For instance, I have an Alert that is the query [site:nytimes.com "Daniel M Russell" Google ] -- that way I can follow anytime my name is mentioned in the NYtimes. With just a few site: limited queries, I can pretty much track just the sources I'm interested in following (and thereby avoiding all of the other Dan Russell's in the world). Note that I use the long form of my name since I know that's the writing convention style of the NYTimes (they always use the long form of your name). At other sites I might use a slightly different query.

I might do an Alert query such as [site:.edu Daniel M Russell ] to track any mentions of my name on an EDU site. This is awfully handy. Since I give a lot of talks at universities, I can track anything published on an academic website that mentions my work (e.g., student posts after I give a talk there).

Note you can vary the kind of content that's searched (web, news, video, blogs, discussions, books) and vary how often it's fed to you (as-it-happens, once-a-day, once-a-week).

Now, what I'm really interested in... is how YOU use Alerts. Do you have them set to monitor some particular aspect of your work? Personal interests? Also, how important are the Alerts to what you do?

I know that in some cases, military families set up Alerts to monitor any possible news items about their loved ones. (See NPR's recent story about the Darkhorse Battalion, and their use of Alerts to watch for possible news stories about their sons and daughters in Afghanistan.)

If you're a teacher, a library or journalist--well, ANYONE who uses Alerts--I'd love to hear your story.

Search on!

Answer to "Who is she, and what about that hotel and that chef?"

2012-01-13T07:03:00.000-08:00

The short answer: Well... there IS no short answer, but she's Margaretha Zelle, aka Mata Hari; arrested at the Hotel Plaza Athénée, where the executive chef is Phillipe Marc, who is of French citizenship. The Hotel was used in an episode of Sex and the City, with a scene of Carrie Bradshaw stepping out onto the balcony wearing a black and white dress.

That sounds simple, but read the following story...

When I started this Search Challenge, I thought it was going to be pretty straight-forward. I knew that Margaretha Geertruida Zelle-McLeod had a complex life, but a straightforward death. Arrested by French police, tried under suspicious (and not especially legal or honest circumstances), then executed by firing squad.

But I didn’t know that Margarethe’s story of her life as Mata Hari would be so interwoven with myth after her death. Though I should have known better...

To pursue this challenge... It’s pretty easy to look up Margarethe Zeller and find that her stage name was Mata Hari. (Don't get lost at this step: She has another interesting interesting life to read about!)

But when you start looking for the story of her arrest, it quickly becomes tricky.

I read the Wikipedia entry on Mata Hari, found the name of the hotel, and then second-sourced that version of her arrest story elsewhere. This is when things started to get tricky.

Version 1 (English Wikipedia): Mata Hari is arrested for suspicion of spying on February 13, 1917 at her room in the Hotel Plaza Athénée. That’s the EN Wikipedia version. Many texts agree (e.g., the book “Mata Hari” by Sam Waagenaar (1965), and several others)

Version 2 (French Wikipedia): BUT... the French Wikipedia entry says: "Six semaines après son retour de Madrid le 13 février 1917, le contre-espionnage français fit une perquisition dans sa chambre de l'hôtel Elysées Palace sur les Champs-Élysées (actuellement siège de la banque HSBC France).” ("Six weeks after her return from Madrid February 13, 1917, the French-espionage made against a search of her room at the Hotel Elysées Palace on the Champs-Elysees (currently the headquarters of HSBC France).")

That would indicate that she was arrested at the Hotel Elysées! Ooops! The English and French Wikipedia entries disagree.

There’s support for this elsewhere. According to Shipman’s book (Femme fatale: love, lies, and the unknown life of Mata Hari, 2007), Mata Hari had, by the time of her arrest “moved out of the Plaza for the cheaper Castiglione hotel and then to the even cheaper Hotel Elysée.” The Mata Hari museum in the Netherlands (a place with, presumably no axe to grind) says that “On 13 February 1917 she was arrested in her room at the Elysées Palace Hotel.”

There is lots of support for both versions--both hotels seem to have credibility as the arrest location. The question is, which one is right? To further confuse things, there are other variations.

Version 3: The BBC has it that she was captured on a train near Paris.

I can go on like this for a while. (I don't want to tell you how long I spent reading about Mata Hari.)
It all gets a bit funny when you compare the different accounts since it’s clear that her story has had lots of.. um... enhancement over time.

A recent article in the New Zealand Herald is typical reportage about her life:

“In February 1917, a French judge and a dozen police officers barged into Suite 113 in a luxurious hotel on the Champs Elysees [the Elysee Palace]. The beautiful female occupant appeared - naked, according to one account - and handed around chocolates in a captured German helmet.”

Surely, Mata Hari didn't really need 12 police officers AND a judge to arrest her. There's something more to this story than just a simple espionage arrest.

In other versions of the story, she was either naked at her arrest AND execution, blew a kiss to the firing squad beforehand, or was wearing an elegant gray outfit that she had made for the occasion.

You know things are getting whacked when some stories have her in Room 131, while others have her in Suite 113. She was nude either at her arrest, or her execution, or both.

As they say, we must go deeper. The only way to resolve this is by looking for the original arrest report. How are we going to get THAT?

It turns out that the original report is quoted in the book, Mata Hari: songes et mensonges, by Fred Kupferman (2005). He wrote this from France while looking at original documents.

From the official police report :

"La fille Zelle Marguerite, dite Mata Hari, habitant au Plaza Palace Hôtelp, de religion protestante, née en Hollande le 7 août 1876, taille 1,75 m, sachant lire et écrire, est prévenue d’espionnage et de complicité d’intelligence avec l’ennemi, dans le but de favoriser ses...”

Okay. It’s the “Plaza Palace Hotel,” which I take is the same as the current Hotel Plaza Athénée.

NOW... we can figure out the rest.

Or.. can we?

When I wrote this question, I'd done a search and found that Alain Ducasse was the Executive Chef. After all, there's a restaurant at the Hotel Plaza Athénée with his name on it. His own website mentions that he "entered the Hotel Plaza in 2000."

A quick check on Wikipedia for Ducasse's biography, and--voila!--we find the newspaper article saying that he switched his citizenship to Monaco for tax reasons.

I'd been stung by the difficulty of Mata Hari, though, so I triple-checked my source with a simple query: [ executive chef Hotel Plaza Athénée ] That led me to the hotel's own website, http://www.plaza-athenee-paris.com/restaurants-bars which says that he's the executive chef.

Except I did a little more clicking around on the hotel web site...

I ALSO found this page on the Hotel web site that says Phillipe Marc is the executive chef... and that this position was given to him by Alain Ducasse!

Wednesday Search Challenge (1/11/12): Who is she, and what about that hotel and that chef??

2012-01-11T07:04:00.000-08:00

In our continuing series of remarkable women in remarkableplaces at remarkable times, meet Margaretha Zelle.

She was a bit of a free-spirit, widely known for herexoticism, her willingness to be au naturel at a time when that was a tad scandalous, and she was equally well-known for her terpsichoreantalents.

She had a bit of a complicated passport,born in the Netherlands, but living for some time in Asia, and visiting much ofEurope.

Unfortunately, she was arrested for treason at a hotel while inParis, tried and then executed by firing squad.

Today, at this same hotel, the current executive chef also has acomplex passport. He’s a bit of acelebrity chef who is also well travelled and has an exotic country as his home. But he has not, so far as I know, posed au naturel, but I have no doubt that he has served Asperges au Naturel in his fine dining establishment.

Our question for today: What is the citizenship of the current executive chef at the hotel whereMargaretha was arrested?

It's a bit of detective work to follow the chain. But it's of medium difficulty. Be sure to say how long it took you to figure it out!

For extra credit, what TV series used this famous hotelfor a set?

And for BONUS extra credit,what color dress was the lead actress wearing when she stepped out onto thebalcony and first saw the Eiffel Tower?

Search on!

What is AND about, really?

2012-01-10T18:40:00.000-08:00

One question I hear quite a bit is this: Don't we have to teach the basics of Boolean search to our students?

The answer, from a Google perspective is this: We teach Boolean searching only for using traditional database systems.

Here's why.

Google queries let you use OR freely between terms. It's basically a way for you to control your own synonyms. A search like [ "mountain lion" OR cougar habitat ] will look for [ "mountain lion" habitat ] OR [ cougar habitat ] and then rank order the results. Synonyms like "puma" or "painter" will be pushed farther down in your results list. In effect, your query is pulling up those synonyms to a higher place in the results. You'll still get terms like "puma" in the results, but they're farther down on the list.

So, OR basically tells Google that these terms are synonyms. Note also that parentheses are dropped from the query. What this means is that you group your OR terms together (e.g, [lungs OR pleural OR respiration systems] without using complicated sets of parens. Terms that are in an OR list (e.g, [a OR b OR c w OR x OR y OR z] ) are all synonyms for terms within that list. Thus, a b c are all synonyms, while w x y z are all considered as synonyms for each other.

Realistically, I use ORs when I have particular synonyms in mind that I want Google to use. For example: [ "high pile" OR fleece OR polarfleece jacket ] asks for 3 different synonyms for the same concept... the ones I want Google to use.

What if I don't use OR?

Then you're implicitly ANDing the search terms together. Except it's not really an AND.

So then, what is AND?

For Google, AND is basically a no-op. That is, it's just another word that you can search for--it doesn't affect the way the query is handled at all. You can see this for yourself. Compare the differences in the results of these queries in the image below:

#1 [ screening injury ] #2 [ screening and injury ] #3 [ screening AND injury ]

You can see that #1 (with no AND or and) is searching for documents that have both the terms in it. That's an implicit AND. If it were a Boolean AND, then both terms would HAVE to be in the document. Thing is, for other searches (say, [ xeric redemption plangent VXII ] ), then you'll get pages that may not have all of the search terms on that page, but might have synonyms or other variants of the terms. If you want "Verbatim Search," you can get that (see my post on Verbatim)

And if you compare results #2 and #3 above, you'll see that the term 'and' is just another search term in the query. That's why it's bolded in the 3rd result of panels #2 and #3.

Make sense?

To summarize: OR gives you specific control over the synonyms that are being searched for; everything else is implicitly ANDed together. Google will try its best to find documents that have all of the search terms in your query, but it will try synonyms and spell-corrections in an effort to do what you really meant (but only after everything else has failed).

I don't know about you, but this "trying other queries after everything else has failed" approach has saved me on multiple occasions. Google's synonymization is pretty extensive; it's part of what makes the search results so robust.

Search on!

Answer: Who found the lost Roman eagles?

2012-01-05T05:16:00.000-08:00

The short answers to our questions from yesterday:

1. Who found the three lost eagles? This is a bit of trick question.

Bust of Germanicus

Two of the eagles were found by Germanicus in 15/16 AD. (aka Germanicus Julius Caesar who lived from May 24, 15 BC until October, 19 AD) and the third eagle was found by Publius Gabinius in 42 AD, more than 50 years after it was lost.

2. Someone gave one of the eagles’ finders a posthumous honor in September. What was the honor and who bestowed it?

The emperor Caligula did many things to honor Germanicus (who was, after all, his father), but the only honor that was "...in September" was to rename the month of September to Germanicus. Obviously, the change didn't stick long term, but it WAS a September honor! (And it's worth knowing that changing the names of months to honor someone was fairly common in Ancient Rome; September was changed to Germanicus twice by different emperors!)

Extra credit:

Who was the eagle finder’s wife? Answer: Germanicus' wife was Julia Vipsania Agrippina (aka Agrippina theElder, who was also a cousin on his mother's side). She was the second granddaughter of the Emperor Augustus, sister-in-law, stepdaughter and daughter-in-law of the Emperor Tiberius, mother of the Emperor Caligula, maternal second cousin and sister-in-law of the Emperor Claudius and the maternal grandmother of the Emperor Nero. Roman family trees are complicated, but Agrippina was about as well-connected as they come.

Now... how did we find these?

I, like most of the people who wrote comments, found that Germanicus recovered two of the eagles. That was a pretty easy find if you do a search like [ three roman legions lost eagles ]. That pretty quickly takes to you articles about the "Battle of the Teutoburg Forest" or the "Varian Disaster." (Note that was did the search with the word "three" rather than the numeral "3"--it's more likely that authors would write "three" rather than "3" -- especially in the context of "three lost eagles.")

And once you've found this, most articles also comment that TWO of the eagles were recovered later by Germanicus. (And if you're not reading carefully, you might think you're done! But no, there's a third eagle to find.)

I then did [three legions' eagles lost in 9 CE] (here I DID use the numeral, since I'm looking for a date) which led me to an article about Roman military history with an important contrast between the different stories about the eagles' recovery. That article told me something useful: the names of several classical historians, including the name of Cassius Dio.

Now that I knew this, a search for [ Cassius Dio eagles ] led me to Book LX of his his text, "Roman History." In his writings I did a Control-F for "eagle" and got this this bit of prose: “…Publius Gabinius conquered the Cauchia and as a crowning achievementrecovered a military eagle, the only one that still remained in the hands ofthe enemy from Varus' disaster.”

Like many of you, I also got swept up into the story of the Varian Disaster and Germanicus' attempt to recover the eagles. It was a remarkable time, filled with remarkable people.

For the extra credit step, it was important to keep all of the various people named "Germanicus" (there are at least 3 important ones), the various people named "Tiberius" or "Nero" or "Livilla" and "Agrippina." It doesn't help that they keep changing their names during the course of their lives!

This wasn't a hard search so much as one requiring attention to detail. Note that Germanicus (who found the eagles) was "Germanicus Julius Caesar" and that his wife was "Agrippina the Elder" (Julia Vipsania Agrippina) not to be confused with Agrippina the Younger (her daughter), or Marcus Vipsanius Agrippa (her father).

Search Lesson: There are a couple of things to take away here:

1. Be sure your question doesn't presuppose your answer. When I asked for the "finder of the eagles," don't assume that there was only ONE person who found them. (Several readers found that Germanicus had found the eagles, but didn't notice that he didn't find them all.)

2. When dealing with complex family trees, be sure you've got the right person. Roman genealogy and history is notoriously tricky. What we say about "always getting a second source" is especially true here. Whenever possible, use the original form of the name (e.g., "Agrippina Major") rather than the transliterated equivalent. (Why? Because the transliterated form can be done in various ways: Is "Agrippina the Elder" the same as "Julia" the same as "Old Agrippina"?)

3. When dealing with history, be aware that there can be multiple answers. This is always true (see, for instance, our story about "Who found the first oil in California?"), but it's REALLY true for Roman history. Not only are there variant versions of the history, but there are a fair number of them. The Romans loved to write, and everyone has an axe to grind or a patron to flatter.

4. When searching for dates, using numerals (e.g., [ 9 AD ]) works well. When searching for quantities (e.g., [ three eagles ] ) you typically want to use the written-out form of the number, since that's the way most authors will write it.

For the most part, people seemed to get the answer! Great job. Next week will be a bit tougher! Stay tuned.

Search on!

Wednesday Search Challenge (1/4/12): Who found the lost Roman eagles?

2012-01-04T08:33:00.000-08:00

Three Roman legions were strung out in a long line atleast 20 kilometers long. It’s been raining and conditions are generallymiserable. The trail through the forest is muddy, everything’s wet, the food isn't great, and lifegenerally sucks for a soldier walking deep in a foreign land on an improbablemission of conquest.

Suddenly, screams and battle cries come from nowhere. Out of the forest flies a vicious attackalong the entire length of the march. Withlight swords, large lances, and narrow-bladed short spears, Germanic tribesmenpound the legionnaires, sweeping in from the forest, hacking and thrusting theirway down the line.

Rome’s finest fightback in a doomed attempt to save themselves. Although the battle takes 2 days, in the end the three Roman legions, around20,000 men, were utterly destroyed.

Even now, 2000 years later, you can still find bits of Romanmilitary equipment at the battleground, scattered along 20 kilometers from oneend of the long line to the other.

In the battle, all of the three Legions' standards, their Eagles, werelost.

Now, with that background image firmly in place, today’ssearch questions:

1. Who found the lost eagles?

2. Someone gave one of the eagles’ finders a posthumoushonor
in September. What was the honorand who bestowed it?

(And for extracredit, who was that eagle finder’s wife?)

Search on!

Search strategies--What are they?

2012-01-03T14:14:00.000-08:00

This month I'll be teaching 6 classes on how to be a better searcher. I'll teach lots of tactics and tips (basically, the tricks you use to drill down to just the bit of information you're looking for)... but I also teach search strategies: that is, the larger guidelines you follow to guide your searching over the longer term.

There are many pieces of search strategy knowledge to have, but perhaps the biggest one is:
Know when to ask someone else for help.

I've written about this a few times before, but it remains true. One of the hallmarks of a truly expert searcher is one who knows when to stop their current search strategy and switch to another strategy. And one of the best secondary strategies is to call someone who you know is an expert in that field.

One of the implications of this is that really great searchers are inherently somewhat social: they have a relatively large network of friends they can tap when they get stuck. And this is one of the truly great uses for social networks such as Facebook, LinkedIn or Google+ When you've hit a brick wall, consider tapping one of your specialist friends. If the problem is solvable, they just might be able to solve it in a few seconds, whereas it might take you hours.

Remember, that when all else is lost, consider one of the "Ask A Librarian" services. (I also wrote about "Ask a Librarian" before, but it's worth bringing it back up in this context. They're often extremely good, and they're always your friend.)

Now, a question for you: What search strategies do YOU employ in your searching?

I'd like to get a nice collection from my distinguished readers. I'll summarize + comment on them in a future post.

Search on! (Strategically...)

Flight Search -- the good parts and the less good parts...

2012-01-02T09:37:00.000-08:00

Google recently launched a new FlightSearch tool. It's a pretty nice thing, featuring very slick interactive widgets for visualizing the options you have (search for nearby alternate, maybe cheaper, airports; tradeoff time-to-fly vs. cost; and so on). It really lets you rapidly explore the options available to you in a much faster way than I've seen on other travel sites.

For instance: If you do a query on Google like [ SFO to ORD ] (San Francisco to Chicago), you'll get a search feature that pops up in the organic search results listing the top/best options for that flight.

Note that the thing in yellow is an ad placed there by the top bidder. (In this case, Virgin America.)
What I'm talking about is below the ad, starting with the first blue link. If you click on that link, you'll shoot over to the FlightSearch tool.

Once you're here, you can look at your options with the visualization tools.

Map Tool: If you click on the Map pin (just to the right of the "Chicago (ORD)" text), you'll get a map of the flight you're searching. Trick to know: If you click on the pins at either end of the flight, you can check out nearby airports. In this case, my starting airport, SFO, is clearly cheaper than the other nearby alternates, so I'll stick with this.

Calendar Tool: If you click on the Calendar tool icon (just to the right of the date line), you'll get a widget that lets you explore the various flight prices and dates. (Watch the video below to see a demo.)

Scatter Plot Tool: This is my favorite as it lets me tradeoff flying time (which I want to minimize) and cost (which I also want to minimize)... With this interactive chart, I can pretty much focus in on what I want to know.

Here you can see that I've moved the sliders (they're labeled "Duration: 5h 30m" and "Price: $244" in this image) to values that I like. You can also see that there are plenty of crazy fares out there (look in the upper right corner: that fare is $500 and over 7 hours... you DON'T want to be taking that flight).

But there are things you should know.

1. No international flights. And yeah, although some might think Canada is domestic travel, Toronto and Vancouver really are international destinations. You currently can't use FlightSearch to find travel to there (or, more probably, given the time of year, the Caribbean).

2. Not all airlines are listed. In particular, neither JetBlue nor Southwest play with FlightSearch (there may be others I don't know about). So if you're traveling to Long Beach (CA) or a place like Tucson (AZ) or Corpus Christi (TX), you might want to check out the Southwest site directly. As far as I know, Southwest also doesn't play with Kayak, Expedia, or the rest of them. (Note: Some people have reported that not all domestic flights of the listed airlines are showing up. I haven't seen that myself. Let me know if you see a gap.)

However, a good trick to know is that a Google search (such as [SFO to LGB] WILL show the airlines that fly point-to-point, even if they don't have individual flights listed... except for Southwest... You have to check them by hand...)

Search on!

Video demo:

How long does it take to find an answer?

2011-12-30T06:52:00.000-08:00

This week's challenge ("Who IS that woman?") wasn't especially tough. It was interesting, but not particularly hard to do. But there are some people who will disagree with me, and therein lies a great tale.

Roughly 12,000 people read the challenge, and if you look at the 150+ comments I got on this, it seemed to capture many readers interest. People who solved the problem reported time-to-solution ranging from 5 seconds to 60 minutes. I went through all the comments (and personal emails) to collect 94 reported times. Here's the chart of how long people said they took to find the answer.

The times on the bottom are in minutes. You can see that 22 people took less than 1 minute (in many cases, less than 30 seconds) to discover that our "fascinating woman" was Lou Henry Hoover, wife of Herbert. Or, to put it another way, all of the people in the green part of the histogram (who took 4 mins or less) were almost exactly half of the total. In other words, 50% of everyone who solved it found the answer in less than 5 minutes. (And yes, I know this is a highly biased sample. It's only people who solved it AND commented on their solution. For all I know, there are thousands of people who couldn't solve it at all, or took more than 60 minutes and were embarrassed to report their times.)

The other way to look at this is that 50% of those-who-solved-it took 5 minutes or MORE (that's the red part of the histogram).

This kind of pair of curves is fairly common in my research. There's a group that gets to the problem solution fairly quickly, and another group who take longer... 50% took less than 5 minutes, while 50% took more than 5 minutes. Another interesting fact is that 14 people in this sample took more than 10 minutes to find the answer. The average time-to-solution was 6.4 minutes, which is, all things considered, fairly long.

What's going on?

There are three things to keep in mind:

(1) Insider knowledge: In a sample like this you always have people who just know the answer or have some kind of "insider knowledge." In the comments I found geologists and mining engineers who actually read the book in school. Another person lived near the Hoover house and had picked up a lot of local knowledge by circumstance. People with inside knowledge can often make a query that jumps right to the answer. These are people who did queries like [ Hoover Latin text ]. *I* certainly wasn't in that group. Maybe you were. But for almost any search challenge I pose, there are people with some info that makes them very fast.

(2) Fast searchers: And then there are the "fast searchers," people who picked out the most unique terms in the problem that let them get quickly to the answer. These folks look for very "individualistic" terms (such as [ female mining engineer Stanford ] ) and manage to scan the results rapidly to find the best candidate solution. The best searchers then double-check that candidate to see if there's a second (or third) source that confirms the insight.

(3) Slower searchers: To be sure, some of the searchers who took between 5 and 10 minutes just got sidetracked by the material. Kudos to them for being curious! But I also know that 16% of the people who answered took more than 10 minutes to answer the question.

The question that interests me is "Why did the slower searchers take so long?" In some cases, it was a problem of making incorrect assumptions. Paradoxically, this often happens to people who are experts in the field. They say things to themselves like "this can't have been a 15th century text, it must have been a 17th century text... let me start there..."

And this leads to a search lesson for today...

Search Lesson: Even (especially!) if you're an expert in a field, do the simple, dumb, obvious search first. Don't dive too deeply into the content without first checking out the obvious and straightforward.

I know this seems counterintuitive, but it's true. (I really know this because it happens to me all the time...) Check the basic queries first, THEN dive down. As I wrote in an earlier blogpost, Don't miss the forest for the trees.

Search on!

Answer: Who IS that woman?

2011-12-29T08:11:00.000-08:00

The short answer: Lou Henry Hoover, who with her husband,Herbert, translated De re metallica(Latin for “On the Nature of Metals” (or minerals more generally)). For this, Lou and Herbert won the Mining andMetallurgical Society of America first Gold Medal for Distinguished Service(1914).

Acomment: Why was this so easy? Well, I was in a generous mood, that’swhy. Think of this question as a holidaygift. I ran across the story of LouHenry Hoover and that “wouldn’t that make a great Search Challenge”? She’s such a fascinating character that Icouldn’t resist. So I wrote thequestion, but then put in TOO much detail. When you’ve got that many unique characteristics, it’s pretty simple tofind the goal. (Next week’s question will be tougher!)

Howto find it: As several readers pointed out, the simplest search to solvethis Challenge is [Stanford woman mining engineer] – if you think about it,there just can’t be that many such people ever in the history ofStanford. And, with that query, you’requickly led to Lou Henry Hoover (her Wikipedia article is the second hit), and onceyou read that entry, all the answer is there before you.

Another approach would have been to work backward from the published work. Several readers searched for [published 16th century Latin manuscript mining ] and found the text in translation, then looked up the translators. That works as well.

One very clever soul actually went to the trouble of converting the image I provided into black&white, then undoing the mirror flip I'd done and THEN doing a search-by-image. I'll have to give them cleverness points, and will be careful in the future to do more subtle edits!

Search lesson: When you havevery distinctive information (female mining engineer, Stanford) or (published 16th century Latin mining manuscript), use those asyour initial key words. You’ll get intothe the answer space very quickly.

Now, about Lou Henry Hoover: Here's a bit of background I can’t resist, she’s such an interestingperson.

Born Lou Henry in 1874 in centralIowa, grew up a bit of a tomboy and developed an early love of rocks, mineralsand mining. At age 20 she enrolled in Stanford’sgeology program, met Herbert, fell in love, but completed her degree. When shegraduated, he cabled a marriage proposal and she cabled back yes. (Possibly the first proposal by what waseffectively “instant messaging”?)

They were married in the CarmelMission (the beautiful mission church in Carmel, CA) and left the next day for miningwork in China. There, they lived for several years, traveling throughout the country, living throughthe Boxer Rebellion, and in the process they both picked up enough Chinese to be able to speakit whenever they didn’t want anyone else to understand what they weresaying! (An interesting aside: Herbert wasthe first-and-only president to speak Chinese, while Lou Henry is the onlyFirst Lady to speak any Asian language.)

While doing some research at the British Museum in London, Lou Henry found animportant classical book on mining, Agricola de re Metallica. It was originally published in Latin in 1556 as a manual of mining andmetallurgy. Lou had been fascinated bythis book since she had originally seen a copy of it in Professor Branner'slaboratory at Stanford and after buying a copy, Lou and Herbert began to translatethe book into English. The translationwas their joint hobby over the next 5 years, dedicating most of their evenings together in the translation process. Lou Henry was the Latin expert, while Herbert knew a great deal aboutthe pragmatics of mining—together they were a great translation team. Their published translation led to their gold medal forachievement from the Mining and Metallurgical Society of America. (See: Lou Henry biography at the Hoover Archives.)

As First Lady, Lou Henry Hoover invitedall Congressmen's wives to visit the White House including Jessie DePriest --the black wife of the nation's one black Congressman, Oscar DePriest. In a timewhen the Ku Klux Klan held genuine political power, this was called "anarrogant insult to the nation” and was an act of real courage.

I have to admit to a deep surprise in reading about Hoover's wife. He'd always seemed to be such a hands-off, do-nothing kind of president, it came as a pleasant discovery to find that behind the history lies some deeply interesting personalities.

Search on!

Wednesday Search Challenge (12/28/11): Who IS that woman?

2011-12-28T07:11:00.000-08:00

Every so often you read about someone who is so interesting and so unexpected that you have to find out more. I spent a happy few hours this week reading about a woman who fit this this description. She was, in many ways, a woman out of time—even though born in the 19th century, I have a feeling she'd be very comfortable living today.

She was a mining engineer who earned her degree at Stanford, traveled to China with her husband at the end of the 19th century and managed to survive the rebellion at that time. In the process she learned Chinese well enough to speak it with her husband whenever they didn't want to be overheard in the US, and with her husband, published the first translation of an important 16th century Latin manuscript on mining. Here's an image from that text (that I've color-shifted and edited in ways that break the search-by-image function. Sorry. I'm not going to make your search that easy!)

There's much more to say about this woman, but if I tell you too much more, it'll be a piece of cake to figure out who she is. But I will tell you one other clue: she was introduced to her future husband by the famous geologist Dr. Branner while they were still in school. They married and had a pretty amazing life together.

The question for you is this: Who is this remarkable woman? For extra credit, what gold medal did she win? (And why?)

Again, if you can, please let me know HOW long it took you to find the answer, and if you can, WHAT process you went through to find out!

Search on!

Answer: Earthquake weather?

2011-12-22T07:54:00.000-08:00

The short answer:

Wednesday, April 18--unseasonably warm (around 80F according to some reports) and clear, but windy. (But other reports say it only got up to 62.)

Thursday, April 19--same as the day before, unseasonably warm and clear, but windy. No temperature data.

Friday, April 20--the usual northwesterly winds returned, cooling off the city and returning to normal San Francisco weather. No temperature data.

Saturday, April 21--temperature data lost. (But probably warm and clear.)

Sunday, April 22-- warm; high: high of 72F, no low temperature data.

Monday, April 23--0.51 inches of rain; high of 60F (SF Chronicle, April 24, 1906; NOAA records)

Tuesday, April 24- 0.14 inches of rain; high of 60F (NOAA records)

While some of our readers found this pretty straightforward, I actually spent around 2 hours trying to get the answer. This was difficult for many reasons. First, it's San Francisco--land of many micro-climates (the weather can vary 20 degrees F across the city; it can be raining in the western part of the city, but sunny in the eastern. Second, because of the disaster, local newspapers stopped publishing and the weather stations in San Francisco were destroyed. (And, as noted, with such wildly varying microclimates, you can't just extrapolate the weather nearby, in Oakland for example, to what was going on in San Francisco.) Third, there was general chaos in the city, and noting down the weather was not first among the priorities when there were so many homeless and the city was burning.

This all made for a great search challenge as it quickly became clear that the normal tricks-of-the-trade (such as reading archival newspapers) wasn't going to work as easily.

I began my search with the obvious [ San Francisco weather 1906 ] and ended up reading lots of commentary that told me about April 18th and 19th. Those days were unusually warm and clear, which meant that the immediate aftermath for the survivors wasn't miserable, their town might be burning down, but people could camp in Golden Gate Park without the cold and misery of fog.

But getting the next couple of days data was harder.

One of the best resources I found (after much reading through search results) was the book The Great Earthquake and Firestorms of 1906: How San Francisco Nearly Destroyed Itself. If you search within this book for "weather" you'll find a good piece of the book describing the weather of the week on pages 51 and 52:

p. 51. “The weatherthat early morning was unusually “clear and pleasant,” according to the weatherbureau. The usual damp early morning fogwas missing. There had been no measureablerainfall for more than two weeks. Aridge of high pressure extended over eastern Washington and Idaho with a lowpressure are ot the south, forcing desert winds to flow from the northeasttoward the central California coast—the reverse of the usual pattern of coolnorthwesterlies that blow from the ocean toward the land.”

P 52. “The the firstthree days after the earthquake, the city was buffeted by wildly gyrating windsgenerated by the changing patterns of weather and the internal workings of thefirestorm. On the first day, theprevailing northwesterly wind and the unusual easterlies dueled for dominance,speaking the flames to the east and thewest. The easterlies increased in tempoon the second day; the fires were driven westward. By late Friday, a strong northwesterly windfrom off the ocean had reasserted it dominance, cooling the city and forcingthe fire back toward the waterfront and across fresh combustibles. On Saturday there was rain, which ended the fires,but discomforted the many refugees sleeping in the open or temporary shelters.”

“The temperatures were ideal for a conflagration. On the second day the thermometer jumped adozen degress to an unseasonable high of eighty-one, as measure in Berkeley andOakland, since San Francisco no longer had a functioning weather bureau. Friday was also unseasonably hot.

That tells us the weather from April 18-21. What about the rest of the week? And would it be possible to be more precise?

This gets tougher. Since many weather stations were destroyed, most of the records were lost. Luckily, the weather service is full of remarkable and pretty persistent people. Alexander George McAdie ( Born 4 Aug 1863, died 1 Nov 1943) was at his post in the Mills Building in the 3400 block of Clay Street and kept meticulous records. (He was also vice president of the Sierra Club from 1904-1913, and was responsible for the naming of Mount Muir.)

His records show (see the actual scans of his log books for April 1906). This is a fascinating document as it's the official temps and precipitation.

You can find the official compilation of all this data by searching for [California "climatological" data 1906] which leads you to Climatological Data California (compiled by the California State Agricultural Society).

But this question started with what the human experience of the weather was. And it sounds from reports as starting out fine, but growing increasingly unpleasant with the cold, the fog and the wind returning after a few days. There's no question that San Francisco can be a chilly, windy place with "the northernly tradewinds adding to the inhabitants misery" on April 20th. (per NYtimes, April 21, 1906)

Search Lessons: A big takeaway from this challenge has been how much data can be found in online Books. Finding the Climatological data book was a surprise, although I ended up there because I saw a reference to it somewhere in my reading. I jotted down a note, and then searched for [California "climatological" data 1906] (the word is in quotes to prevent synonyms from being search--I knew this was in the title of the book).

Another technique I tried, although it was slow going, was searching for diaries and letters written at the time. Again, there's no shortage of such material, but it's slow going to extract weather information from such texts. People weren't especially worried about the weather until it started to rain, so they didn't write much about it. (By the way, most of the letters are absolutely fascinating when you start to see the discrepancies between the official reports of what happened vs. individual's eye witness reports. I'm often skeptical about eye witness testimony, but the variance between the number of dead officially reported and the number of dead bodies observed is pretty striking. The city seems to have massively underreported the casualty rate.)

Additional reading: There's really a huge amount of stuff out there about the earthquake and fire. Many of the stories are tragic, some heroic, and some are just unbelievable. (Some authors believe that the majority of damage to the city was caused by ill-advised attempts to create firebreaks by dynamiting barriers. This worked, except when they blew up a distillery, which caused massive explosions and sent firebrands all over the place.) Here are three that I found especially interesting.

SF Chronicle article about our hero, Alexander McAdie: http://www.sfmuseum.net/press/clips13.html

SF Virtual Museum articles (they have a great wealth of diary and oral histories about the disaster).

Letter from President Theodore Roosevelt to Congress asked for the Department of War to assist in supplying San Francisco with much needed relief. (And, interestingly, a reply from Wm. H. Taft, who became president following Roosevelt, and a few quick notes from General Funston, for whom Fort Funston in San Francisco is named.)

Postscript: The images below are from a set of stereoscopic images I found while stopping in a library in Point Arena, CA, a fairly small and rural town about 100 miles north of San Francisco. I had an hour or so to hang out in the downtown area, so I wandered into the library and discovered a pile of old stereoscope images just sitting out on a display case. So far as I know, these have never been published online before. (Sorry about the relatively low-res, but these were captured with my cellphone camera, which was the only device I had available at the time!)

This was taken on California St in downtown. The car might have been staged,
but the background buildings are clearly completely destroyed.

This scene is from Fort Mason, one of the major refugee sites for survivors.
The Army supplied tents and a thriving mini-city lasted for months afterwards.

Search on!

Wednesday Search Challenge (Dec 21, 2011): Earthquake weather in San Francisco

2011-12-21T07:20:00.000-08:00

The recent Tōhoku earthquake and resulting tsunami on Friday March 11, 2011 was devastating with massive loss of life and property, putting thousands of people out of their homes in an instant.

This made me think back to the last giant disastrous earthquake that happened in San Francisco--the 1906 earthquake that destroyed the city through a combination of ground shaking and subsequent ill-planned attempts to stop the fires that were started. Much of the city was leveled, then much of it burned, leaving at least 250,000 people homeless.

As you know, San Francisco can be a lovely place, but it can also be cold, windy and foggy. While it never really snows, it can be a very unpleasant place to be camping without supplies while the city is being restored.

It's pretty easy to lookup the weather in the Tōhoku region for the period immediately after the earthquake. But today's question is a bit more difficult than that:

What was the weather like in San Francisco on the day of the 1906 earthquake AND for the next 7 days afterwards?

Two views of downtown San Francisco after the earthquake and fires of 1906.
What was the weather like if you were homeless after this earthquake?
(Both photos by George Lawrence, taken by kite photography, 1906.)

This gets to be slightly tricky as all of the communications infrastructure also went down with the earthquake. So, how will you find out what the weather was? Was it cold and foggy? Was it raining? How miserable was it for the survivors?

Caution: When I was searching for the answer to this, I kept getting pulled into remarkable and amazing stories about the earthquake. You might too. This search took me hours longer than this should have because it was a fascinating topic. Careful that you don't spend too many hours researching this amazing time in California history.

Be sure to include a quick description of HOW you found the answer, WHAT resources you used, and HOW LONG did it take you to find the answer.

Search on!

Seeing the gestalt, finding a key term

2011-12-14T06:06:00.000-08:00

It was a 50 yard fog. Beyond that distance, it was all just white, a slowly swirling, pearlyinfinity… the kind of fog that hides everything outside of a certaindistance.

Or does it?

As I walked through Mitchell Park early that morning with mycoffee in hand, I was struck by how different everything seemed. It was quieter than usual—you’d expect thatin the fog---but was I was noticing was how the structure of the park, the veryshape of the open fields, the arrangement of trees, the layout of space seemedsomehow… different… Mysterious, sure, but in particular, the masses of treesseemed somehow more apparent, less ordinary, more.. I don’t know…present.

As I crossed the small bridge over Adobe Creek I sawredwoods looming up out of the translucency. There was one, maybe 100 feet high. I walked another 30 yards and could see more redwoods emerging as Imoved forward.

“Redwoods” and “fog” go together about like “orange” and “juice,”they co-occur all the time. Redwoodslove the fog and rely on it for moisture in their natural habitat in themountains. But this time it was in the park near home. And for the first time I noticed somethingthat had somehow escaped my attention on all my previous visits to the park. The redwoods were planted in a straight lineat regular 10 yard intervals—a picket-line of tall trees. I stopped and stared—how was it possible thatI’d never, ever seen this regular progression before? It’s not just trees in a line along the edgeof the street, these were hundred-foot giants that were marching like Ents intobattle through the fog of war. Seeingthis progression of trees in my local park made me feel as though I’d somehowmissed a major feature of the landscape. This wasn’t just beautiful backdrop, this was a linear forest that I’dsomehow never noticed.

Was it the fog changing my perception and letting me see thegestalt by eliminating visual noise from the scene? One of the fundamental principles of gestaltperception is the law of prägnanz (Germanfor pithiness), that is, we tend to organize our experience in a manner that isregular, orderly, symmetric, and simple. In this way, things that clump together seem to be a “unit” (such as arow of trees), while items farther apart are seen as individuals. I’ll let you work out what the conditions for“togetherness” and unit perception are.

Could it be that I was seeing some kind of hybridimage? You’ve probably seen thesebefore—when you’re close to the picture your eye picks up all the highfrequencies, but as you back away your eye can’t see the tiny, high frequencytwiddly bits any more, just the larger strokes of the low frequencies. That’s how the same picture can look likeEinstein up close, but Marilyn Monroe from far away. Is that what the fog was doing? Somehow attenuating the highfrequencies?

I suppose that what was happening to me is true moregenerally. Different circumstances causesubstantially different outlooks on the world.

I was thinking about literally seeing another person’spoint-of-view the other day as I reviewed a paper about an “embodied avatar,”that is, the paper described an animated picture of a person (the “avatar”)that could be projected onto walls and manipulated by a real person. The article was all about how confusing it wasto the person who was remotely operating the avatar—how they’d mix up left forright, or swap “near” to them vs. “near” to the avatar. That all made sense, it’s just the normalconfusion people have when trying to run something that’s far away from them,when what-they-see is NOT what the remotely controlled device sees,particularly when there’s a coordinate transformation involved.

I’d had some practice doing this with remotely controlledgliders. I’ve built more than a few fromscratch and had the distinctly painful experience of unintentionally swappingleft for right or up for down and driving my laboriously built glider into theground at a high rate of speed. As theysay in the business, I practiced “controlled descent into terrain” because Imomentarily couldn’t figure out if up was down, or left was right… from theperspective of the glider.

I know this kind of thing is common when you’re doingremote control. I’ve spent many happyhours projecting myself into the seat of my remotely controlled glider, slowlylearning the skill of “seeing the world from the point of view of the glider”rather than seeing things from where I was actually standing.

And as a careful reviewer I wanted to write a review thatwould help out the authors. Point themto literature that they clearly didn’t know about.

So I looked and looked and looked and I finally found thatliterature, but it was also a painful process of discovering what the rightlanguage to use should be to discover it.

I tried things like [remote control glider disorientation] and [drivingat a distance] and even [drone flying practice] (hoping to tap into some Air Force research about remote operation of drones in the military.

But nothing was working. Was I completely off base here? Did that literature NOT exist somehow? Ibegan to question my own memory of reading this stuff.

And then I spotted the key term buried deep as I wasscanning an article on the remote operation of deep sea rover arms. The word was “teleoperation.”

Aha! It wasn’t “remote” *anything* -- the term ofart is teleoperation—when I read that, I had thoughts of scientists from the1960s picking up flasks of radioactive substances, and images from my late1960s schoolboy books of science flashed through my mind. But the prefix “tele-“ was the key that ledme to the robotics literature and to the studies of disorientation because ofeye location vs. remotely-operated system displacement.

This kind of “noticed insight” happens often enough thatit’s got to be more than just an oddity. I find it really interesting that such a tiny, barely observableartifact can drive your search to a successful completion. And itseems it’s just true more generally: Smallchanges in your perception can transform everything you see in anenvironment. That’s when I thought aboutwhat I’d seen in the park.

The day after I walked through the fog I went for a run on atrail in the canyons not far from home in the Hidden Villa preserve. It’s a mixed variety forest—some fir trees,mostly oaks of different kinds, a few big tooth maples, some California buckeyes. Along the trail grows mostly ferns, somehazels, coffeeberry bush, sycamores—the typical understory of the Santa Cruzmountains along streams and rivers. (It’s called the “Central Californianriparian” ecosystem.) In the summertimethe plants are pretty much a uniform green—you can see the leaf shapes, subtlevariations in green, but not much else. It’s pretty much all a wash of plantlife.

But now, in late autumn, the maples have been transformed tobrilliant yellow and stand out like chrome yellow beacons in the background ofoaky green. That’s the kind of thing youexpect. What I didn’t expect were thechanges to other plants visible on Hidden Villa trail. Like the maples, different kinds of shrubshad changed colors, making them visible as newly apparent groups as well. The willows had changed, the poison oak, thesycamores… between all the color changes and leaves falling, very differentkinds of plant structures were visible.

Now, another gestalt principle is the Law of Common Fate—birdsthat fly together in a flock share very similar trajectories over time and canbe seen as a group. In the same way, asyou run up the canyon, bushes that share a color, or are now visible because ofleaves no longer in the way can be seen as sharing a common motion against thebackdrop of the evergreens. And that’swhen I understood what I’d seen in the fog.

I’d initially thought that fog was just adding a white-gauze-likefilter over the scene, and that THIS was what had changed my vision of theredwoods. So, being an experimental kindof guy, I went back to the park, stood at the same place where I’d suddenlyseen the redwoods in a cluster and took a picture with my digital camera.

Then, back at home, I played around for a bit by addingtranslucent layers (representing the fog) at different opacity levels. Here’s the strange result: no matter what Idid, I couldn’t get the redwoods to “pop” out of the background the way I’dseen it on that foggy morning. What didI not understand?

My run up the canyon with the changed foreground against theevergreen background gave me the insight I needed. To get that “pop” gestalt effect, I needed tosee the redwoods as sharing a Common Fate; that is, that they’d appear to movetogether as a unit. But just adding in atranslucent layer of white wasn’t cutting it.

What I realized now during my run was that seeing adifferent background altered what I could see in the foreground. Not only was the fog putting a white glazeover everything, but the fog was ALSO cutting off all of the background behindthe redwoods. It was a screening effect,changing what groups I could perceive by eliminating the background.

When I photoshopped out the background—as though it werebeing done by a 50-yard fog—it became clear what was going on. The redwoods stood out as repeated unitsmarching down the edge of the park.

As I looked at the altered picture, I also realized thatthere were a line of sycamores in front of the redwoods. (They're the small, bare trees between the camera and the redwoods in the picutres.) But they’d all lost their leaves as well. If I’d walked this way on a foggy day inJune, I’d have never seen the redwood gestalt—it would have been hidden behindthe canopy of the sycamores. It wasliterally a case of not being able to see the trees for the forest.

And although it's a little hard to see in a static image like this, when I was walking through the fog, it was clear that the redwoods were supposed to be planted in pairs (at either end of two horseshoe pits--for 4 pairs of redwoods). But as this diagram shows, two of the trees were missing.

The effect was driven by the parallax shift on the two existing pairs of trees (solid lines, dark green) and was so striking that I walked over to the horseshoe pits and found, sure enough, that two of the eight trees were missing-in-action, having fallen prey to time and disease, leaving just stumps on the ground.

So it turned out to be two small changes: one small changein the foreground (the leaves that had fallen, revealing new views) and a smallchange in perception of foreground/background made possible by the fog cutting off the background.

But like running across “tele-“ in my search, my little observation (understanding the effect of fog as creating a moving backdrop obscuring what’sbehind) led to a whole new way of looking at the urban forest, and anappreciation for new ways of seeing the world.

Bottom line for search: There are two big lessons here.

(1) A good searcher often learns from what's on the page, even when it seems off-topic. You can often pick up terms of art that WILL help you focus on what you want to find. In the case of the redwoods,

(2) Sometimes removing the background clutter can improve your vision. In this case, the fog was giving me a background removal function for the visual image. In looking at search results, you'll sometimes have to figure out what to ignore to let the big picture stand out clearly.

Answer: Where is that trail?

2011-12-12T06:56:00.000-08:00

Quick answer: One end of the trail is here, near 13456 S Fork Ln, Los Altos, CA 94022 (or, lat/long: 37.376390, -122.16103). And after reading through some of your comments, I was able to find the other end of the trail on Edgerton Rd.

Here's what I did to find it.

I kept thinking it was a "trail" so I kept searching for a trail map that had "Wallace Stegner trail" on it.

That turned out to be a bit of a dead end. It's partly because Los Altos Hills calls their system of trails "pathways" rather than trails. I picked up that useful factoid by reading about Los Altos Hills trails and stumbling across an article that mentioned their "system of pathways."

Once I did a search for [ "Wallace Stegner pathway"] and it returned only 5 hits (and one of which is broken with a 404 error). Luckily, the link to SF Gate article about Wallace Stegner's house being demolished was good enough.

The key piece of data to take away from this is this sentence:

"It [the trail] starts on Three Forks Lane where the narrow path alongside Matadero Creek near the Stegner property. "

Elsewhere in that article it gives the Stegner home address at 13456 South Fork Lane, Los Altos Hills, CA. (Note: Wallace passed away a few years ago and the residence has changed hands.)

If you look at the Google map for that street address, you can see that the obvious nearby watershed is named "Matadero Creek." And if you use Google Streetview to "walk" down the street to the place where the creek crosses the road, you'll quickly find that location, here.

Click on the image to see the full resolution.

Now, to tell the truth, even though I was THERE, because of the complications of the hills where I'd been running, I really didn't know where the official start of the trail was. So this result surprised me.

Now, my next task... to find the other end.

Since I know that the trail is part of the official city "pathway system," I just did [site:losaltoshills.ca.gov wallace stegner pathway ] to limit my search to the official city publications and that gave me the city newletter, "Our Town" from 2009. In that newletter it says:

"...Wallace Stegner Path from Three Forks Lane to Edgerton Road, named for the world famous author who lived for many years just uphill from that path."

It wasn't hard to use StreetView to find the other end on Edgerton Rd.

Click on the image to see the full resolution.
Note that the first part of the sign was blurred-out by the algorithm
that blurs out license plate numbers. For some reason, it thinks part of
this sign is a license plate.

To be sure, I tried following the trail on the Maps view, and I even tried Google Earth, hoping to get better resolution. But the trail is obscured by heavy trees, and I couldn't follow it reliably. Luckily, the city put up those handy markers to let me find it!

Now... I have to point out that many readers found the Wallace Stegner Memorial Bench located up in the Long Ridge Open Space Preserve at lat/long: 37.278778, -122.160767 It turns out that I've gone running up there too, and it's NOT on the Wallace Stegner trail (or pathway), and it's definitely NOT in Los Altos Hills (as I said in the original challenge). It's actually on Long Ridge Road (a fire road / trail).

It's a beautiful place too... and for some very nice photos of the Stegner Bench, see SmugMug photos.

But I congratulate you on finding a truly wonderful place to go for a hike that perfectly captures the sense of Stegner's writing. If you're in the area, I highly recommend a visit--both to the bench and to the trail.

Search lesson: (1) Check your work. (It wasn't in Los Altos Hills!) And (2) when you're reading through related texts, be sure to stay alert to the possibility that what you're searching for might be called something else. That is, your terminology might not quite be the right one. Stay open to other ways of saying things.

Search on!

Wednesday Search Challenge (Dec 14, 2011): Find that trail!

2011-12-12T06:45:00.000-08:00

I may as well admit it: I'm a runner, and I love to run on the trails in the Santa Cruz mountains near home. It's one of the great reasons to live near the Googleplex--there are wonderful hills to run not very far away. Here's one not so far from home.

Last week I was running near here on a hilly trail just west of Mountain View in the city of Los Altos Hills. They have a wonderful set of trails that wind all over the place, up hill, down dale, and all over hither and yon.

Sometimes I'll go running and just see where I end up. It only takes a turn or two before I can find a new trail.

That's what I did this past weekend--I started running and made a few random turns, ending up on a trail that I'd never seen before! What I *did* find was a sign-post that said "Wallace Stegner." Aha! I was on a trail named for the great writer of the American West!

I turned around, and ran back the way I'd come, and an hour later was back home trying to find the start of that trail on a map.

It took me a while, but eventually I was able to find it.

Can you find the Wallace Stegner trail starting point? (There are apparently two streetside entrances, but I was only able to find one.)

The correct answer will be either a street address or a lat/long. (Just saying which street it's on isn't enough detail.)

When you post your answer in the comments, be sure to say HOW you found it and HOW LONG you took to find it!

Search on!

Answer: Who was she, and how much did she get?

2011-12-08T12:11:00.001-08:00

The questions were: Who was the developer who lost Twain’s investment… Whowas the woman who sued him… and ultimately, how much did the developer payto her to resolve the accusations?

The short answers: Mark Twain invested overmuch in the charismatic developer / inventor James W.Paige’s typesetting invention and lost all of his investment. Paige, a fascinating scoundrel if there ever was one, became engaged (maybe) to Jessie Hall, who later sued him for $950,000, which is a lot now and an incredible sum back in 1892. But the story is so complex that the lawyers eventually signed a statement that discharged the suit with no money changing hands.

I started with by first findingwhat Mark Twain’s “financial difficulty” was all about. The query [Twain financial problems] led to the Wikipedia page about Twain. Then a bit of reading and it was clear that it was the Paige typesetter. (Aka the Paige Compositor)

The Paige typesetter (from www.TwainQuotes.com)

That led me to learn about thePaige typesetter. As noted on Twain's Wikipedia page, "His technological fascination with the Paige typesetterhas often been cited as the ill-fated investment that bankrupted him." Okay, this is the probably the issue we wantto focus on.

So, I searched on [Paige Typesetter invention] and that leads straight to the Twain Quotes Page, a page rich with content, reallyinteresting pull quotes from a number of contemporary resources about Paige, covering all of his foibles and various pecadillos.

Now... one of the operatingprinciples you have to take on as you do internet research is to check everything. So I started checking things.

This page has a bunch of newspapercitations, so this gave me the chance to follow-up and verify that the referenceswere up-to-date and correct. Sure enough,all the citations checked out. (Example: here’s the LA Times archive for one of the citations) Note that you can buy the originalarticle for $2.95. Similarly, you cancheck out an article about the law suit (Jessie Hall, an Actress, Claims Big Damages for BlightedAffections) in the Hartford Courant (June 10, 1892) at -- oddly here, it’s $3.95 (seems expensivefor 244 words…)

The big search point is thatnewspaper archives are often a great way to follow up these kinds oftrails. BUT they often need to besearched newspaper-by-newspaper, which often brings up the problem aboutfinding archives of newspapers that might be out-of-print. You usually need to search for the newspaper,THEN find an online database that has that newspaper.. for the dates that you’reinterested in.

In this case I started with theLATimes.com because I knew that they had searchable archives (throughProquest). I did that search, found thearticle, and then used that to spot check the other articles (e.g., theHartford Courant). Of course, theNYTimes has a free archive, and a quick search on their archive [ "Jessie Hall" James W. Paige ] (in 1892) leads to a nice article in the New York Times. Notice that I used double-quote around "Jessie Hall" to avoid the spell-corrector trying to convert "Jessie" into "Jesse."

This is a great search strategy toknow: Use newpaper archives, but knowhow to find the archives in the first place!

Wanting to check out everything, Ialso went to Google Patent search and, armed with the full name “James W. Paige”and knowing he invented a kind of typesetter, I was able to find his originalpatent, US 547859, "Machine for setting, distributing and justifying type."

Now we're in the place to answer the questions.. But first, the story in short:

James Paige took a good deal ofTwain’s money and then more-or-less wasted it, losing his investment.

Somewhat later, Paige took aninterest in actress Jessie Hall (stage name: Dorothea Lewis) and convinced herto move in with him in Hartford, CT. Apparently all was not well in Hartford as she later had him chargedwith assault and battery of her (and her mother). Then, the way these things go, one weeklater, Paige and Hall left Hartford for Chicago. (!) More disagreements followed and Hall returned to Hartford and filed suitfor $800,000 for promising to marry, but then backing out of the deal.

Just to add a twist, when the casecame to trial, it turned out that Jessie Hall had JUST been to that same courtfor illegally living with another man, Edwin M. Grant.

It’s a complicated set of lives,and Paige seems to have been quite a guy. As Twain later wrote about Paige in his autobiography:

“Paige and Ialways meet on effusively affectionate terms; and yet he knows perfectly wellthat if I had his nuts in a steel trap I would shut out all human succor andwatch that trap till he died.”

So… our last question was “And howmuch was she paid?”

In the end, the lawsuit againstJames W. Paige (Case #S-140309) was filed in the Superior Court of Cook County,Illinois. On February 17, 1893, both the attorneys for Paige and Hall signed adocument dismissing the case without cost.

I’ve thought about getting a copyof that document. If you’re interestedin this, here’s the Archives records search form for the Cook County Courtsystem. http://198.173.15.34/?section=RecArchivePage&RecArchivePage=6015

But it’s a $9 minimum + $2.00 forthe first page and 50 cents per page after that.

Search lessons…

1. Newspaper archives are a wonderful resource for tracking down complex stories, but they're scattered all over, and over are not indexed by Google, so you need to actually use their web sites to search their archives.

2. Newspapers often charge for access. Luckily, if you shop around, you can often find less expensive versions of the content.

Now, was that hard?

Lots of people got this challenge correct, which is great. Interestingly, it seems to have taken somewhere between 2.5 minutes and 20 minutes!

I know that most people consider 10 minutes to be a fairly long task (most search tasks take less time). But did you find this difficult? Is search task difficulty the result of just the amount of time you take, or of the number of searches and clicks you have to do to get to the answer? What's your opinion?

Search on!

Wednesday Search Challenge (12/7/11): Who was she and how much did she get??

2011-12-07T06:53:00.001-08:00

Regular readers of SearchResearch know that we are sober, quiet, and interested in the deeper uses of online research. One typically doesn't find much that's salacious here.

But for today's Search Challenge we'll make an exception. This search is about gossip and somewhat tawdry rumors that we want to run down. The reason this is interesting to readers today is that it involves Mark Twain and a bit of 19th century scandal mongering.

As you probably know, Mark Twain suffered from a bit of a financial setback in the middle of his life. This led to his regular touring shows where he would lecture and generally wax wise about the state of the world.
Today's search challenge is this:

Background: Mark Twain lost much of his money by investing heavily in the development of a very complex device. The developer was a bit of a scoundrel who not only lost Twain's investment, but was caught up in a series of accusations and counter-accusations with a noted woman.

Question: Who was the developer; who was the woman, and ultimately, how much did the developer pay to her to resolve the accusations?

Image from USPTO of the invention that led to financial setbacks for Twain.

Did you know.. Weather in Google Maps?

2011-12-02T10:55:00.001-08:00

I think I'm going to start a new header in my blog posts "Did you know..."

Even though I work at Google, I'm still sometimes surprised by features that launch that somehow slipped by me.

Here's one of them: You can now turn on the "Weather Layer" in Google Maps. Just go to the Satellite / Map icon in the upper right corner of the map, roll your mouse over it and a popup layer will appear. Select "Weather."

When you do, a bunch of weather icons (current weather) will appear as a layer on the map. Here's one for the current weather on the Big Island of Hawai'i.

And here, the nice little demo video the Maps people made.

Glad I finally noticed!

Search on!

Answer: Antarctic Islands and India?

2011-12-01T09:09:00.001-08:00

Here's the short answer:

WHO: Yves-Joseph de Kerguelen de Trémarec (in 1772)

WHAT: Kerguelen Islands, deep in the southern Indian Ocean

WHAT IS CONNECTION WITH INDIA: Kerguelen's rocks are very similar to ones found in northeast India near the Rajmahal Traps (those hills I mentioned). The traps were formed by a geologic hotspot that initially formed in Gondwana. As the plates shifted over time, the hotspot emitted a series of eruptions that left underwater ridges and plateaus over time. As the surface shifted, this hotspot gave birth to both the Rajmahal Traps, underwater features like the Ninety East Ridge and the Kerguelen Islands.

Now, the longer answer... How did I search this one out?

When I first heard about this problem, all I knew was that the discoverer was 18th century French. A few searches like [ French privateer island discovery ] I finally found that it was the Kerguelen Islands (after a few false starts--there were, after all, other islands discovered by the French--Réunion, Mayotte, etc.).

When trying to find out what the connection was between Kerguelen and India I read an article about the Kerguelen Plateau. That's the microcontinent ("once covered in conifers") that's now underwater 1 - 2 km underwater. That got me to thinking about possible geological connections between the plateau and the striking ridges in the Indian ocean.

From there it was a quick search for [ Kerguelen Plateau ] which led me to the Wikipedia article about the plateau. A bit of reading about the Kerguelen hotspot, a quick finding that the Kerguelen hotspot has produced basaltic lava for about 130 million years and has also produced the Kerguelen Islands, Heard Island, the McDonald Islands, and the Ninetyeast Ridge. The very last line of that article suggests a connection with India.

Following a search for [ Kerguelen India hotspot ] leads pretty quickly to a nice article at PhysOrg.com detailing the connection between the two. Here's my simplified diagram:

As the plates move around above the hotspot, a line
of volcanoes and basaltic flood plains (what become

plateaus) forms in a line above.

And as reader Hans pointed out,this is wonderfully documented in the article "Ar(40)/Ar(39) Geochronology of the Rajmahal Basalts, India, and Their Relationship to the Kerguelen Plateau" (J. Petrology, v43, n 4, p 1141-1153, 2002)

Bathymetric map of Indian Ocean (from SRTM30_plus via Wikipedia).

Note the striking ridge (Ninety East Ridge) pointing nearly due
north from Antarctica to the Bay of Bengal.

Postscript: I think I the challenge was harder for me than for you! When I started, I didn't even know where the islands were! Just figuring out that they were in the Indian Ocean took me 15 minutes. So when I wrote this question, I gave several people a head start--they cleverly shortcut the whole search problem with a query of [ Antarctic Islands India ] which gets you to the Keguelen Islands pretty easily. Well, as I say, it's an open-internet search challenge.

But in the next few weeks we'll make the problems a little more difficult--until we're ALL super searchers!

Search on!

Wednesday Search Challenge (Nov 30, 2011): Antarctic islands and India?

2011-11-30T06:56:00.001-08:00

Here’s a tale that I’ve been trying to track down. Put on your best discoverer’s hat and see ifyou can figure this one out as well.

I was told that not long after the Seven Years War, a Bretonprivateer discovered islands near Antarctica that he believed were theheadlands of a new and previously undiscovered continent. That didn’t pan out, but he did find islands that are somehow related to an entiresubmerged small continent that wasformerly covered in pine trees. Strangelyenough, that watery sub-continent is in turn somehow connected to the large hillsnear the city of Rajmahal in India.

WHO found WHAT islands, and WHAT IS their connection withIndia?

You don't need any particularly tricky search techniques for this, but you do need to be willing to look around and find connections that you might not have thought about.

Search on!

Doodles, cool and archived

2011-11-28T20:19:00.001-08:00

I saw a really surprising search the other day. I was watching a school-age kid trying to find one of Google's clever interactive logos. You know, like the one from today celebrating Bob Noyce's 84th birthday.

You've probably noticed--they're sometimes interactive, they're always celebratory in some way and they have a popup explaining what it's all about. (I'm showing the popup on the right-hand side of the above image.)

What you might not know is that if you click on the image on the Google Home Page, it will do an appropriate search for you on that topic.

What was so interesting about watching this kid search for the Google Archive was that he didn't know they're called "Doodles," so he was fooling around searching with queries like [Google images ] (which won't get you very far because the results are so heavily skewed by Google Images, the search verical). He tried things like [cool Google images ] before I finally told him to try adding the word "doodle," which of course got him what he wanted.

Search lesson: Sometimes you just need the right word to access the right body of information. The deep trick is knowing which is the right word.

In this case, the kid might have tried to search for [Google images ] on Images and he would have found all kinds of results, including links to the Google Official Blog, which then has links to the archived Doodles.

To save YOU the trouble, here are a few links to Google's collection of great Doodles.

The official Google Doodle archive

The marvelous Doodle ode to Stanislaw Lem

The Les Paul interactive Doodle (play your own tunes!)

And last, but not least, the video of the famous Charlie Chaplin Doodle.

Search on!

Changes... always changes to Google products

2011-11-28T09:55:00.001-08:00

You probably saw the recent Google blog post about sunsetting different products. http://googleblog.blogspot.com/2011/11/more-spring-cleaning-out-of-season.html

Here it is in short: (Remember, this is my personal blog and doesn't represent Google official thoughts, policy or inclinations...)

1. Google Bookmark Lists (Dec 19, 2011) -- essentially nobody was using them (I mean, not even I was using them, so you know it didn't have much uptake). Fatal flaw--what user problem was it solving? Nobody really knows, so it's going away in December.

2. Google Friend Connect (Mar 21, 2012) -- being superceded by Google+ features.

3. Google Gears (Dec 1, 2011) -- it was a valiant attempt to make Google products work offline as well as online. But with the advent of HTML 5 (and the various offline features it offers), Gears is rapidly becoming redundant.

4. Search Timeline (Oct, 2011) -- this is unfortunate, as there really isn't anything else quite like it. Yes, you can use Google Insights for some of this function, but the ability to do Timelines over News Archives is just gone. I'm hunting around for a good replacement for the ability to do this kind of search + charting.

5. Google Wave (Mar 30, 2012) -- I can't say that I'm sorry about this one being turned down. In my use of Wave, it was just a giant, unwieldy thing. Nice idea... but it was too much all in one package. I want a speedboat, but got a cruise ship.

6. Google Knol (Oct 1, 2012) -- This too was a great idea--authored Wikipedia style articles. But it never took off in the way that Wikipedia has, partly because the articles never got enough links to make them show up high in the results. Sigh. Unfortunately, Wikipedia is suffering its own set of difficulties. (Have you tried to add a new article to Wikipedia recently? See Danny Sullivan's recent rant about this. He's totally right. His critique shows the growing problems that Wikipedia is having, and worries me about its future.)

Moral of this particular story: As I've been saying for a while, things come, things go. The good news here is that Google is getting better about letting everyone know about these changes. You can see the culture change over time. Used to be that changes just happened without any kind of comment. People noticed, or they didn't.

Another kind of change that went unremarked (but I'll tell you) was that there were recently a bunch of changes to the online Dictionary. When you do the [define: ] operation (such as [define:peruse]) the quality has gone up, primarily in marking typical uses. While keeping up with shifting definitions is a full-time job, the fact that Google is paying attention to the quality is a great thing.

But it's one of those things that isn't, from Google's point of view, worth mentioning. It's just one of the continuous changes that tries to make everything better. Of course, not every change is 100% improvement for everyone, but the goal (and I think Google pretty much succeeds at this) is to make the majority of people happier and have access to higher quality information. (For even more information, see Matt Cutts recent post about recent ranking changes or the YouTube video about ranking changes below.)

If you see something that is clearly worse for you (or better yet, worse for a large number of people), let me know. The least I can do is to pass your observations along.

Search on!

Memories of green lines on the land

2011-11-25T05:03:00.001-08:00

There are lines on the landscape that you can’t see easily,lines that mark the memories of stream pathways and old property boundaries,leaving a persistent impression of the past on the surface of today.

We’ve all seen lines of trees before. Depending on the place, straight tree linesare usually old property boundaries, with the trees marking off the edges ofephemeral property ownership that changes at decade speed, while the treeschange at century speed, out of sync with the short / fast / furious lives ofthose that would use them to mark a bounding box around what they think theytemporarily own. In California there’sbeen a long-standing tradition of using eucalyptus trees to mark off edges,even though the trees remain long after the human claimants have been returnedto the soil as nutrients.

What’s even more remarkable is that these green lines can beseen through Google Maps. The aerialimages (“Satellite view,” even though many of the images at the scale we’reinterested in are actually taken from planes) often reveal the past of a placethrough the green lines.

Click on image to see at full size.

In the above image you can see a long line of trees just below the lake on the left. That's Felt Lake in the Stanford University foothills. But those are 60 foot tall eucalyptus trees that used to mark the southern end of the old Stanford farm. You can pretty clearly see what the boundary was. That line of housing development crowds right up next to the property line, making a nearly vertical slash through the hills.

But I’m really more interested in the ways memories of streams persist in the landscape. Here’s an aerial image of some land in Palo Alto taken in 1948. The sinuous line of trees running through the photo is Matadero Creek. The straight line running almost vertically is a railroad track, and you can see that the line of trees following the creek stops at a major highway—in this case it’s El Camino Real, an old road that has been in that location for hundreds of years.

From 1948... I pieced these together from Google Earth images in History view mode.

What captivates me about this is that the tree lines persist for a very long time, even through 50 years of intense development. As you can see, the line of trees along the creekbed have survived, in a sense, for half a century. The creek is in a very different setting now; once it enters the industrial park, the stream is completely driven by runoff from acres of asphalt, and no longer fed by trickles from grassy fields.

This is a recent image. I’ve highlighted in red the creek once it enters the Stanford industrialpark and flows into suburbia. It’s really broken only in two places—firstwhen it crosses Foothill Blvd (just before entering the industrial park), andthen just a bit later when it was channelized to go beneath the Tibco building(the straight segment in the middle of the industrial park).

Seeing these images makes me think of how much of a lastingimpact we have on the world. Developerschannelize creekbeds, but the lines still persist. It’s hard to undo that much geography, it’sjust incumbent unless you move the entire landform.

These lines of trees (oaks, mostly) explain something yousee from time to time walking through suburbia. Every so often, there’s a tree that is way out-of-size, something thatjust seems incongruous with its surround. If it’s an oak, chances are good that it’s part of a much largerstructure—a persisting line of trees that remembers a world before thedevelopers came.

I find this both heartening and sad at the same time. I feel lucky to have come along at a point intime when one can still see these marks on the land. At the same time, it’s a bit like finding apalm tree deep in a scrub oak woodland—the marks of people on the land areinescapable, and often woefully out-of-place.

We cover, we bury, we replant… and still the land manages to persist. The big oak in the suburbs is an oddity, wayout of keeping with the other trees in the area, until you realize that it’sbeen there for 200 years. Ultimately,that makes me glad. I hope the trees winin the end.

And feeling lucky enough to live in a time when I can see both the trees from the ground and from the air through Google Maps and Google Earth.

Searching for more trees and more green lines on the land...

Answer: What's that flower?

2011-11-24T04:59:00.001-08:00

The quick answer is that it's Ithuriel's Spear, aka Wally Basket, aka Grass Nut.

Latin name: Triteleia laxa.

Regular reader Hans nicely provided a link to an illustration of Ithuriel and the spear he used to reveal that the frog he's threatening was actually the devil in disguise. (Who knew?)

To solve this challenge... I first looked up where I (were the flower) was. If you enter the lat/long into Google Maps, 37.1540, -121.4200, you'll find that you're in the middle of Henry Coe State Park.

Why does this matter? Because when locating things like wildflowers, birds, animals, insects (etc.) the geo-location matters a great deal. There are many thousands of blue wildflowers in the world, but your first big clue in figuring out which one it might be is location.

Once I knew that this flower was in a State Park, I figured that this might be useful in doing my search. So I did:

[ blue wildflower Henry Coe State Park ]

I could have used a search term like "northern california," but I went with Henry Coe because I know that state parks often have volunteer organizations that publish things like collections-of-wildflower-photos.

Once I did that search, I saw that one of the top hits was "Album of Blue Wildflowers from Henry Coe State Park."

Ah ha! Since I had a couple of good pictures (see yesterday's challenge page), it was pretty easy to compare with each of the photos in that album. And, sure enough, the 13th image down is a match.

But matching flowers based on just an image can be tricky, so I copied the Latin name (Triteleia laxa) and did another search on just that.

[ Triteleia laxa ]

which led me to the Wikipedia page for Triteleia laxa (which wasn't all that helpful, but did give me another image to compare). It ALSO led me to the Calflora site entry for Tritelia laxa, which IS a great, very authoritative site. (Why is it authoritative? Because it links to very well-known herbarium sites such as the Jepson Manual and the USDA plant manual.)

That pretty much confirmed it for me, but to be triply sure, I ALSO checked the Jepson Manual entry for Triteleia laxa... which was consistent with my field observations about number-of-stames, descriptions of anthers, flower composition, etc.

A quick search for [ Ithuriel's Spear Milton ] confirmed that the story of Ithuriel's spear and the devil-disguised-as-frog appear in Book IV of Paradise Lost by the 17th century English poet John Milton.

Search moral: There are a couple of takeaways.

1. When searching for animals, plants, and similar localized things, consider starting your search with a geo-reference. In this case I used "Henry Coe State Park" -- I could have used something else (even California as a search term would have helped.)

2. When looking for a specific flower, you probably need to find images to verify your observation. If you REALLY want to get into the details, you're going to need to use a wildflower identification key to verify that the flower you're looking at really is what you think it is. Note that most keys are region specific--be sure the key you're using is for the location the flower is in!

Search on!

Wednesday Search Challenge: What's that flower? (Nov 23, 2011)

2011-11-23T17:23:00.001-08:00

You've probably asked a question like this before: What's that flower?

Unless you're botanically inclined, or have a friend who's horticulturally minded, it's often kind of hard to figure these things out... especially for wildflowers that are unrelated to anything you'd see in a domesticated garden.

As you might guess, this happens to me all the time.

This past March I was out for a leisurely hike at lat/long 37.1540, -121.4200 when I noticed a pretty blue flower. Here are a couple of pictures so you can get an idea of what I saw.

It was about 3 feet high and was growing in open grassland--nothing especially odd or strange about where it was growing. Just along the trail, sticking it's head up

When I got home I looked it up based on what I saw.

To make your task slightly easier, I'll tell you that the flowers were blue, but that there were some variation in flower color between the plants. They were blue, blue-purple, or white. The flower tube was around 12–25 mm long, with the petals ranging from 8–20 mm in length. The flowers are cone-shaped and are made up of 6 petals that are fused together at the bottom. In the flower there are 6 stamens, 3-lobed stigma atop a single triple-chambered ovary. In the flower, filaments attached at 2 levels, and the anthers ranged from 2–5 mm. As you can see, individual flowers were arranged in clusters of 4 - 8 flowers at the end of longish stalk, roughly 100 mm long. The leaves are long and grass-like in appearance.

When I figured out what it was, I was surprised to discover was that its common name is the name of a weapon that's used by a particular angel! (Do angels normally carry weapons??)

Question: Can you figure out what the Latin and common names of this flower are?

For extra credit on this one--be sure to write down HOW you figured this one out, and how long it took you. I'm curious to see if you find this one hard or very simple. Let me know!

Search on!

Intuition and counterintuition

2011-11-22T05:00:00.001-08:00

Intuition: What’s the deal with intuition these days? It seems to be on everyone’s mind, abrainworm on the loose. People areclaiming that tablets like the iPad “tap into intuition,” that Steve Jobs hadan “intuitive designer’s sense,” George Bush “trusted his gut feelings” aboutthe presence of WMD in Iraq, Kim Kardashian decided that “intuition led me todivorce” after 72 days of wedded bliss, and the Huffington Post writes “Science saysto trust your gut.” Intuition seems tobe more valued than ever, although there seems to have been no recent upgrade inour collective intuitive skills. Is intuition really just the flowering of some inner, secret power?

By contrast, the current movie “Moneyball” is about thesuccess of the Oakland A’s team of 2002, a team that was put together withguidance of some clever statisticalanalysis by their general manager, a baseball quant-jock if there ever wasone. Even Roger Ebert says the film isabout “the war between intuition and statistics.” Is there really a war going on?

In his book “Blink,” Malcolm Gladwell talks about intuitionas “thin-slicing” experience based on training up your experience base after 10,000 hours of practice in a field. If you have that much time-on-task, practicing and learning how to operate efficiently in a field, then you’recapable of rapidly assessing a situation based on few, rapidly scanned clues,and somehow coming up with a quick recognition of what’s going on.

Here's the thing to know: How much time have you spent doing search? My best estimate is that I've done about 5,000 hours of search since I first started using Google in late 1998. That is, I started practicing my search skills around 13 years (or 4748 days) ago.

If I've done a bit over 1 hour of search / day since then (which seems reasonable), that means I've invested ~5,000 hours of practice. While that's a lot of time, it's only half of the 10,000 hours that are usually needed for real expertise.

System 1, System 2: Nobel Prize winner Daniel Kahneman points out that people have twodifferent and parallel systems of thought when they confront problems. The cleverly named “System 1” is a fast recognizerof situations in context—it identifies and labels objects, picks up onrelationships, and does so by recognition, rapidly, rather than bydeliberation.

Then there’s “System 2,”the slower, more deliberate, symbol-pushing and rational part of ourminds.

As an example, System 1 recognizes that the number sequence2, 4, 8, 16, 32… is just doubling from one to the next. It would be System 2 that lets you realizethat this is also the powers of 2. Of course,if you’re a computer scientist, the powers of 2 has become, over many repeatedexposures, something that’s a System 1 effect. For non-CS majors the problem 2 * 256 is a System 2 task. For CS-majors, it’s a System 1 task—yourecognize the pattern and say, “it’s 512…” without thinking much. In this sense, intuition is what you’ve beentrained to expect to perceive. It is thepower of repeated exposure and the accumulation of inarticulate recognitionskills.

About System 1 Kahneman writes in his book Thinking, Fast and Slow:

“We are prone to think thatthe world is more regular and predictable than it really is, because our memoryautomatically and continuously maintains a story about what is going on, andbecause the rules of memory tend to make that story as coherent as possible andto suppress alternatives. Fast thinking is not prone to doubt.”

Nor is System 1 particularly good at noticingcontradictions. We swim in a sea ofcounterintuitions--things that seem to be intuitively correct, but are not. We have difficulty seeing the sea we swim in simplybecause we swim in it all the time.

Some examples:

Howis it that clouds made of water vapor yet can float mid-air? (Water is awfully heavy.)
The world is visibly and obviously flat—yetwe now believe that it’s intuitively obvious that the world is round. (Trust me, historically speaking, that wasn’tobvious at all!)
When we’re on amerry-go-round, our intuition tells us that the force is outward—thatcentrifugal force is really trying to thrust us radially away from the center,not on a tangent along the direction of travel.
Dense things are typically opaque, except for glass andwater, which are “intuitively obvious” exceptions to the rule.

Counterintuitive: Except that they’re not obvious, it’s just the pattern you’ve seen so often that System 1 doesn’t even pick up on the contradiction.

Intuitive thinking is primarily what you’ve experienced andon patterns you pick up. While it’sfast, it’s also errorful. Dan Ariely has asked hundreds of Princeton undergraduates the following question: “A bat and ball together cost $1.10. The bat costs $1 more than the ball. How much is the ball?” It’s asimple question, but around 50% of Princeton seniors get it wrong and say thatthe bat costs 0.10. (That can’t makesense. If the ball is 0.10, then “$1more than the ball” means that the bat is $1.10 – adding the bat and ballprices together means the bat + ball are $1.20.)

So, why do they get it wrong so often?

Because they don’t check their work. Why? Because it’s a bit of a hassle, and the answer is so obvious andapparent (and the stakes are so low) that it’s not worth the effort. This is characteristic of many intuitiveanswers—it’s so *obvious* that it’s not worth checking.

Seems to me that there’s not really a contradiction betweenSystem1 (intuition) and System 2 (rational) modes of knowing. Instead, one’s just faster than theother. Both are useful, both arenecessary. But the great achievement ofscience has been to tell us that intuition always has to be checked. In essence, science is the overturning ofintuition by slow, painful, careful System 2 reasoning. Clouds float because cloud-borne waterdroplets are really, really tiny and are kept aloft by random air movements.

A note from Dan’s System 2: Cloud droplets average between 1 and 100 microns. A typical droplet 20 microns in diameter is4.2 picoliters in volume with a weight of 4.2 nanograms, falling at a terminalvelocity of 0.02 mph. Thus, an updraftat just over 0.02 mph will keep the cloud in the sky. That’s not much updraft. Or to look at it another way, if the cloudformed at 10,000 feet, at 0.02 mph it will take nearly 4 days to fall to theground.

The trick is to know when to use one mode versus theother. “Trust the force, Luke” Obi-Wanwhispers into our ears. But the sametime, you can’t build a pan-galactic empire with massive infrastructures basedon your intuitions about mechanical engineering, power grid design, or wastemanagement systems.

We know there are a number of rather rapid inferences thatare System 1. A while ago Bob Zajoncshowed that determining preference is much faster than cognition… that is, itanticipates cognition. You see this whenmaking choices about which kind of thing you prefer over something else—be itmates, brands of peanut butter or random Chinese characters. As he said, “Preferences need no inferences.” It’s fast and gets therebefore you’ve figured out what you really should want. That’swhat System 2 reasoning is for.

It strikes me that we have a lot of brainware dedicated tospecific fast System 1 type tasks, and even those mechanisms get it wrong muchof the time. We’ve all seen variousvisual illusions that we KNOW can’t be true—a vertical table that looks muchlonger than a horizontal table; two patches of color that look very differentin a scene, but can be easily shown to be the same color when placedside-by-side.

The strange thing is that visual illusions persist, evenwhen you know they’re wrong; even when you’ve measure the differences; evenwhen you put the color patches side by side. Even though you do the measurements yourself, you can’t convince yourvision system to accept your measurement as truth. It’s as though you haven’t learned anythingin the last minute of experience.

(In the example below, squares A and B are the same color. Use an eyedropper tool to measure the colors and find they're the same. This illusion image is from Wikipedia, created by Edward H. Adelson, Professor of Vision Science at MIT in 1995.)

Even though we DO more vision than anything else we do withour brains, we still have predictable,repeatable mistakes in vision. How canwe be better in domains where we don’t have so much hardwired brain?

This is the realm of the cognitive illusion, and is evenmore problematic. We know that if youset up a program with an opt-out vs. an opt-in choice, the differences inselection rate are huge. In one Europeanstudy, the rate of people selecting to be in a organ donation program wasmassively different: in Sweden the participation rate is 86%, while in Denmarkthe participation rate is only 4%. Thedifference is that one is opt-in while the other is opt-out.

Dan Ariely points out that when we’re filling out the formwe

“feel as though we’re making decisions… but the person who designs the form thatis actually making the decision for you. You like to think that the options don’t influence us, but that’s nottrue.”

The illusion is that you havecontrol over your destiny. That is, youthink your System 2 rational decision-making system will actually express yourdeeply-held, fundamental beliefs. “Ibelieve that organ donation is a social good, so I’ll participate.” That’s the story you might tell yourself. But statistically speaking, the choice ismade for you when the form is designed as opt-out.

We don’t recognize the illusions we all live withday-to-day, we’ve come to accept them as intuitive and obvious. Normal is what you’ve grown up with, thenon-intuitive, confusing, difficult high-technology world is whatever was inventedafter you turned 8, and stopped seeing everything as pre-destined.

Isn't that intuitive?

Verbatim mode - Google without interpretation

2011-11-18T04:32:00.001-08:00

Remember the ruckus about the + operator going away?

We knew about the issue, and this week Google has announced a new search mode that you might want to know about.

Verbatim mode was announced on the Inside Search blog (a blog I recommend you read, actually, since most of the interesting announcements about new Google search features will appear there first).

What's Verbatim search? It's Google search without any synonymization, spell-correction, personalization or other interpretation. That is, it's just a basic search without any alterations in what you typed into the query box. To be clear, 99% of the time, those alterations actually improve your search results quite a bit. But every so often you really want just what you typed. Verbatim mode is for those situations.

To turn on Verbatim, you first open the "More search tools" button on the left hand panel...

That is, click on the bottom option, just below "Custom range..." This will open up a new set of options below that point. It should look like this:

When you click on Verbatim, you'll put your subsequent searching into this hyper-literal mode.

As we've discussed before, sometimes a search term seems so obviously misspelled (to Google) that it can't resist spell-correcting it for you.

But as you know, in this case we REALLY want to word to be unaffected by spell-correction or synonymization. You already know that you can double-quote the word like this:

... to get the Finnish folk music style that you were really searching for.

Now, when would you use Verbatim?
Whenever you want to do longer queries that you don't want altered at all.
For instance, a multi-word query such as [ joiker music finland ], a trip to Verbatim mode might be exactly what you want. Here's the query in default search mode:

But you might want to search this in Verbatim, as below. The first result is the same, but the next several results are somewhat different.

Notice the blue bar at the top of the results.. that's how you know you're still in Verbatim mode.

The difference is probably easiest to see in a side-by-side like this. Here I'm searching for "gyros":

As you can see, the Verbatim mode search (on the right) gives a very different answer--one that's NOT localized or personalized.

Hope you find this new tool useful! Let me know how you like it!

Search on.

Answer: Where's the quarry?

2011-11-17T04:39:00.001-08:00

The fast answer to the question: the Tennessee pink marble for the Newseum First Amendment tablet comes from the Endsley quarry, which is in Friendsville, TN at 35.75413, -84.1334

Here's what I did to find it.

I did a straight-forward search on [ Newseum stone ] which led me to this article in KnoxvilleBiz.com

"...The marble came from the Endsley Quarry in Friendsville and was provided by Tennessee Valley Marble before the company was purchased last year by Tennessee Marble Co.

Architects chose Tennessee pink marble for the tablet because the same stone was used on the exterior of the National Gallery of Art, which is across the street, and in several other prominent buildings and monuments throughout the city..."

Wow. THAT was easier than I thought.

I did a big of digging into the Tennessee Valley Marble company and found that it actually runs several quarries, but that the Endsley quarry, just outside of Friendsville, wasn't used for many years, but was re-opened for the Newseum project and is still active.

By using the "Drop lat/long marker" tool in Google Maps, it was pretty simple to find out the lat/long.

Along the way I found that they needed only 50 tons of stone for the tablet, but in order to get enough decent stone, they had to quarry out (and discard?) around 278 tons.

If you look to the right (east) of the quarry, you can see a stone cutting facility that you can't see from the road. But if you zoom in, you can spot large blocks of stone sitting around outside, looking like small cars.

But if you look closely and use the measuring scale in the lower left of the image, all of the blocks at the quarry are actually fairly large. They're not at "finished size." Most of the blocks in this image above are around 10 feet on a side.

It looks to me as though they're quarried here, cut in to large blocks, then shipped off to the main cutting facility in town at the Tennessee Marble Co, on West Vinegar Valley Rd. (I found this location by zooming the map out a bit and doing a query for [ Tennesee Marble Company ].)

At this location a few miles away from the quarry, you can spot many more slabs that are of the right size to be sent off for final engraving and installation. You can watch the video of the engraving and installation by the Rugo Company which did all of the work of actually putting words onto the stone. In this video you can see that the individual slabs are really fairly small--the kind of slabs that would be precision-cut in a place like the one above.

As several of you geologically inclined readers pointed out, Tennessee Pink marble actually isn't marble, but a kind of crystalline limestone that looks a lot like marble when polished. As the Wikipedia article on marble points out: "Geologists use the term "marble" to refer to metamorphosed limestone; however stonemasons use the term more broadly to encompass unmetamorphosed limestone."

So Tennessee pink is really just unripe marble. It needs a bit more metamorphosis to mature from limestone into marble!

All the comments this time were great! Many readers figured this one out. Excellent. We are getting smarter!

Search on!

Wednesday search challenge (Nov 16, 2011): Where's the quarry?

2011-11-16T08:43:00.001-08:00

I'm in Maryland this week and had the chance to visit the Newseum in downtown Washington DC. It's an incredible museum of news, a place I highly recommend when you visit the DC. It has a great view of the capital building and on the top floor, a wonderful vista of the entire mall.

While I was there I spoke with man who told me that the beginning 45 words of the First Amendment to the constitution are engraved on giant slabs of marble on the front of the building. He went on to say that it was made of the same stone as was used to build the museum directly across the street. "The idea was that they wanted it to match, so they went and re-opened the original quarry."

He didn't know where the quarry was that was used as the source of stone, but he was pretty sure it was "down south" somewhere.

I was curious about where such a massive chunk of rock would come from, so I searched it out. It took me about 3 minutes to find an aerial photo of the quarry from whence it came.

I now know where the quarry is. Can you find that aerial image and provide the lat/long of the quarry?

For extra credit, can you find the lat/long of the place where the stone was cut to shape?

Search on!

Answer: How do people spend their time?

2011-11-11T08:00:00.001-08:00

Whaddya know? This turned out to be simpler than what I thought--for an unexpected reason.

Here's what I did. My first query was to see if I could find any database that had reasonable information about how Americans spend their time day-to-day.

[ Americans spend time ]

This query led me quickly to the American Time Use Survey at the US Bureau of Labor Statistics (BLS). This is their job: collect data and stats on American behavior across many different dimensions.

I drilled down almost immediately to their data sets about time use and found data from their big survey of ~13,200 Americans in 2010. The metadata for this survey is here (which is where they write down all the details of how the surveys were coded, the survey question form (ever want to see the script that surveyers use when asking you questions--this is it), etc.

Eventually, I got to the summary table of "Time spent in primary activities (1) and percent of the civilian population engaging in each activity, averages per day by sex, 2010 annual averages." I exported that data into my spreadsheet, and generated the following chart rather quickly.

Whenever you look at a chart like this, questions and insights immediately spring to mind. For instance, WHY is the average amount of work / day only 4.09 for men and 2.94 for women? (Or 3.5 average between the two.) Or, to answer my original question--"Do I spend a lot of time in email?" The answer is "yes, you do... but you're in Silicon Valley, what did you expect??"

You might prefer to see the data in this format:

(Same data, just different chart.)

Remember that this is a sample of people nation-wide, balanced across demographic categories (age, gender, location, etc.) and included a representative sample of unemployed people as well. Keep in mind, if you work a standard 40 hour work-week for 50 weeks of the year, you're really only working 4.3 hours/day averaged over the entire year.

Contrast that with the number of hours / day if you're engaged in that activity--7.82, which reflects more what you'd expect for number-of-hours worked given that you're working on that day.

We could continue analyzing the data, which is really interesting, but I want to return to the search question for a minute.

Why did I say this was simpler than I thought?

Answer: Because I had a particular solution path in mind when I started out. I would (1) locate the data, (2) extract it, and (3) analyze it.

I never expected that the BLS would have already done this for me! As gasstationswithoutpumps commented,
"I searched for [ hours spent american statistics ] which got me to
http://www.bls.gov/news.release/atus.nr0.htm which has pointers
to tables for 2010 statistics divided in lots of ways. (in particular...)
Table 12 http://www.bls.gov/news.release/atus.t12.htm
seems to be the one you want, with sleeping and watching
TV as the two biggest categories."

I never expected that they would have already done the analysis, with charts and everything! (Note that this chart from BLS is slightly different than mine--they're charting the time use on an average workday for employed persons with children. The chart above is for everyone in the sample set.)

Now....

On the other hand I read analyses of how people spend their time as compiled by various other organizations. Neilsen, for instance reports that people spend 4.9 hours/day watching television. That's very different than the amount that the BLS finds in their data. You can find similar differences for other measures as well (e.g., internet use; amount of time spent care-giving; household activities; ....).

The big takeaway is that you have to be VERY careful that you know exactly what data is going into your analysis. It matters how it was collected, what the coding scheme was, and how the sample was chosen. Was it random digit dialing (as in the case of the BLS data), or was it "Neilsen Family" people? The outcome can vary tremendously.

One of the beautiful things about being able to find data of this kind is the ability for us to download and analyze it according to our own interests. The folks at FlowingMedia did a beautiful interactive version of this same data by building on D3, Mike Bostock's visualization system. The image below is just a single frame from their system. You should visit the FlowingMedia interactive visualization (click on the demographic segments in the upper right) to get the full experience.

Bottom line: Searching for a data that you can easily download, understand and do your own analysis of is a great feature of the new Data Age. You should learn how to get to data AND how to understand it. That metadata stuff turns out to be deeply important.

Yours in finding ever more data!

Search on!

Search Challenge (November 9, 2011): How do people spend their time?

2011-11-10T06:35:00.001-08:00

It might not look like much, but this is a compelling graphic:

It's a chart of how people in the US spend their time during an average day.

Can you figure out how people spend their time? You can probably guess what the big orange and big blue segments are, but what makes up the other 30% of the day?

This question came about from a simple discussion at work the other day. I know that I spend about 2.5 hours each day just doing my email. Was that normal? Is it a high number? A low number?

So if you take the US as a whole (statistically speaking), can you figure out how normal people spend each day?

The best solution here is to be able to create a chart something like this one (although I'll show a different graphic tomorrow). You need to find data you believe--a source that's credible, or at least one that uses some kind of data-gathering mechanism that you believe.

Be sure to tell us how you found your data and why you believe it.

Search on!