Tuesday, January 24, 2023

A personal note: Transitions

 I didn't expect that.

Lock image by Eyeball3000

When I tried to badge into my workplace, into my Google building at 4:30AM last Friday morning, the badge just didn't work.  No click, no dice. It was dark, cold, and very, very quiet.  

Earlier I'd tried to login to Google corporate gmail, but all I was seeing was some very suspicious pages saying "access denied" and that I should login at a site I didn't recognize with a password I didn't know.  Very, very odd. 

So I'd gone into the office to file an incident report using the trusted office network.  When my badge didn't make the door pop open, that's when it hit me: I realized that what looked like a phishing attempt was actually my company telling me that I didn't work there any more.  

Well... that's a surprise. I didn't expect to be locked out without some notification.  

As you've probably heard by now, Google had a massive layoff on Friday, Jan 20, 2023, and I was part of that reduction in force action. 

I really don't have any explanation about why I was laid off.  Somehow, I was determined to be on the CUT side of the team management equation.  So be it.  It's just odd because I would have sworn that I was doing core research work on the intersection of search and the new Large Language Models like ChatGPT. It's really interesting technology that will heavily influence the way search works.

So the real question for us, our little SearchResearch community is this: What now?  

As I've mentioned before, this isn't a Google corporate blog--it's our blog.  I write about search engines, about curiosity, and research.  I also write a lot about Google, but also about other systems and methods of doing search.  I'm not beholden to anybody here. So we're free to continue if it makes sense.  

So.... I'm trying to figure out what I'd like to do next.  What makes sense for me, and what makes sense for SRS.  

As you know, I'm working on another book (running title: "Unanticipated Consequences").  If you remember back to the very first SRS blog post, I pointed out that creating book content was always the goal of SRS--to pull together enough interesting material to write something meaningful.  And we did that with The Joy of Search.  (Coming out in paperback soon!!)

The question for me is this: Do I keep spending the roughly 8 hours/week on SRS, or do I switch to UC (Unanticipated Consequences)?  

Short answer:  For the moment, I'll keep on writing here on SRS. I have a long list of potential Challenges ahead.  I'll do at least 4 or 5 more of the really fun ones, and then we'll figure out what's next.  

So, count on SRS posts for at least the next little bit.  I'll poke my authorial head back up out of the murky SearchResearch depths in a few weeks and chat about what the plan is going forward.  

In the meantime, keep posting all of your great comments and ideas to the blog.  I read absolutely everything y'all write, although I only have time to reply-to a few.  I love the activity, the questing minds, and the conversation.  Let's keep that going!  


Search On!  

Wednesday, January 18, 2023

SearchResearch Challenge (1/18/23): Musicians travels--how did they get from A to B?

 I'm a classical music fan... 

Bach and Chopin, both travellers. But how did they travel?
P/C Wikimedia sources.

... and I've read more than my share of books about the musicians that I admire.  But as you also might expect, I'm curious about some of the details of their lives--little things that make me wonder, How did they do that?  What was their lived experience actually like?  

Most recently, as I was traveling through central Europe, I was thinking about two famous trips made by musicians and trying to imagine what it was like to travel in those days.  Now, of course, you hop on a train and can be across Europe in a matter of hours.  But it hasn't always been thus.  

So I did a bit of SearchResearch on two very famous trips and learned some remarkable things.  Let me pose them to you as an SRS Challenge.  When I answer this, we'll talk about doing historical research like this--what works, and what doesn't.  

Our Challenges for this week are: 

1. Frederic Chopin traveled from Paris to Majorca (also spelled as Mallorca) in 1838 with the hope of improving his health. It was a disaster from beginning to end, but as I was looking at a map of the Mediterranean, I wondered about he got there--it's not exactly around the corner. How did he travel to Majorca?  Obviously he took a ship, but from where? And how?  How long did it take? Now I'd just take a ferry, but was there regularly scheduled service in 1838? How did he travel back when he returned in 1839?  

2. Johann Sebastian Bach also had a famous trip that left me wondering about the details. In 1705 he traveled in the winter from Arnstadt (where he was living and working) to Lübeck to hear Dietrich Buxtehude play the organ at his parish church.  That is also not exactly around the corner. It's nearly 400 km (248 miles)!  How did Bach get from Arnstadt to Lübeck and back?  How long did it take him to travel? 

So many questions!  

My goal in posing this Challenge is to get you to think about how to approach such questions.  Where do you turn first?  How do you dig into the content?  

I hope you'll find this one engaging and fun.  I honestly didn't know the answers, despite all of my reading on the topic. I know now, and it amazes me about what travel was like back then.  Perhaps it will amaze you as well. 

Search on! 

P.S.  I'm trying to get back on schedule by posting this Challenge today.  I'll give the second part of the answer to last week's Challenge later this week.  

Monday, January 16, 2023

Modifying a Reverse Image Search Query: Google Lens and Reverse Image Search

 I took a photo, but can't remember where it's from... 

Happens to me all the time.  

Naturally, I'll use Reverse Image Search (also known as "Search by Image") to find out what it is.  

But as you might know, Google's Reverse Image Search has been updated by Google Lens.  The big difference is that Lens tries to identify objects and give you results based on that identity, rather than just finding similar images.  

Often, that's exactly what you want. In this case, I want to know what this image that I found on my (physical) desktop is showing.  

A quick scan, then go to Images.Google.com and search for my image. This is what I see: 

That's a pretty good result: I know it's "Spring" by Lawrence Alma-Tadema, the immensely skilled Victorian classical painter.  (For a nice article about this painting, see the Getty's publication about this work.)  

But if I want to learn more about this painting, including where it's used, I can click on the "Find image source" button at the top of the left-hand panel.  That will open another tab with the previous Search-By-Image results.  Like this, which includes the "best guess" search string that is created by Google, approximating what it thinks I'm searching for: 

And, as we've learned in our earlier discussions of Search By Image, you can modify that query to give you more precise results.  Here, I've modified the query to include a SITE: restriction, this one limits the results to .EDU sites, especially useful if I'm looking for high quality results, such as those from the Getty Museum, where this original painting is housed.  

This ability to modify the query is really useful when you're searching for things that aren't products--such as images of plants.  Here's an example of a nearby creek I photographed during last week's heavy rainfall.  Here's the original Search By Image query (yes, that's a swollen creek near my house):  

But if I'm looking for similar images in my home state of California, I click on the "Find Image Source" button and modify the query to include [flooding in California].  This finds visually similar images that are also connected with the idea of flooding in California

Hope you find this ability to do the classic reverse image search a useful extension.  

Search on! 

Friday, January 13, 2023

Answer: How can I find latest updates on topics of interest?


Staying on top of an emerging topic...  

P/C MidJourney. Prompt: "unanticipated consequences, realistic"

... is a continual issue for professional researchers. 

Often, your work can't be answered and "solved" in a single search session, or even a single day, but must be compiled over a longer period.  This is one of the defining characteristics of complex and sophisticated research problems--they take time.  (See some nice work from Microsoft Research on this: Slow search: Information retrieval without time constraints.)   

But for us, the Challenge is having some way to track emerging results and new insights in a field.  How do we do that?  

As I mentioned, the obvious way is to subscribe to blogs and newsletters that monitor the topic for you.  That's good, but suppose you'd like to get a bit more from the news directly?  

Let me frame this as an SRS Challenge for you: 

1. Can you find a way to limited search over a small number (say, 3 - 7) of high quality periodical sources of information for a particular topic for the past year?  (In my case, I want to search for articles on "unanticipated consequences" during 2022. Your topic of interest might be something different.)  How can I do that? 

I quite liked SRS Regular Reader Krossbow's post, so I'm going to borrow much of his response here.  In particular, he focused on setting up multiple and different kinds of regular alerts from alert services...  

My initial thoughts went to Google Alerts as that's my normal tool when I want get updates on a subject. As you know, it runs your query of choice repeatedly and emails you the results.  Here's the official source for how to set up a repeating alert:  Google Alerts

I also thought about searching social media feeds. Feedly is an RSS reader I've used since Google Reader was shut down. They allow you to set up regular feeds based on search terms, but that capability is in the paid tier.  {Dan: I haven't tried this. If anyone does let us know how well it works out.}

My thinking then went to my old favorite standbys of library research: EBSCO and ProQuest. My library gives access to current periodical corpora as well as ProQuest.com. Login through your university (or public library access point), and you too can create Proquest alerts

 {Interesting side-note: Clarivate acquired ProQuest in 2021--they also run Web of Science and services like EndNote, so things might change in the future. General SRS point: Things change. Stay up-to-date.}  

Another set of resources your library might have access to is EBSCO Search which also allows you to set up alerts.   Creating a Search Alert in EBSCOhost - Tutorial

I searched for [gale books alerts] to find Gale Alerts and RSS Feeds for Gale Books and Authors.  {Looks like paid subscription is required, but this alert stream notifies you when a book with a title matching your query is added.}  

My library uses Libby for some periodicals. If your interest is broad enough, you can set alerts when a new issue of a magazine becomes available.  Set a Libby alert when new magazine issues are available.  

Same for JSTOR (another aggregator indexing service like Proquest).  See JSTOR alerts.  (Again, you need to login here--check our your library for access.)  

I went to SimilarSites (an easy way to find other web sites that are similar to one that you specify) and searched for sites similar to Proquest.com.  This led me to several sites, the most useful of which is probably ScienceDirect has open access papers, journal articles with this tutorial for setting up alerts. ScienceDirect alerts tutorial.

As I read through Krossbow's post, it reminded me that the SemanticScholar website also has an alerts system.  Here is SemanticScholar's alert service.  In practice, it seems very similar to Google Scholar's alert service; both index the scholarly literature, but they have somewhat different feeds and indexing times, so you'll see somewhat different results.  

When setting up alerts with these services, it's important to get your queries right.  For my original topic of interest, "unanticipated consequences" the trick is to find other expressions that will get you the insights you're looking for.  

When I just brainstorm a bit, I came up with these other expressions: 

Original: unanticipated consequences 

Next ideas, search for these phrases as well:  

unexpected consequences
boomerang effect
didn't expect
unanticipated effect

And then I ran out of good ideas.  Is there some guide to help me broaden the search?

Sure!  I did a search for each of these phrases and read a bit in the hits on each search. For example, the search for [boomerang effect] took me to the Wikipedia article on that topic.  A quick scan of this article told me that the phrases "backfire effect" and "Barbara Streisand effect" might be useful as synonyms.  Setting up an alert for each of these (rather than using an OR in a long query) lets me figure out which of these will be productive.  I'll let them run for a few days and turn off the alerts for phrases that don't work out well.  

When I step back to think about what I just did, I realize that I could mine Wikipedia for other phrases just by searching like this: 

     [ site:en.wikipedia.org "unexpected" ] 

(Or, do the same thing with "boomerang effect" "Streisand effect" etc.)  

When I did that query, I ALSO found that the query terms "paradox" "unexpected discovery" or "paradoxical effect" could also be useful synonyms.  You can play this synonym expansion game forever, but don't.  I suggest you get a few, run a test search to see if the results are what you need, and then set up an alert with them, one at a time.  After a week, keep those alerts that are useful; prune the rest.  

One other idea... When I'm doing research like this, I often find it useful to search for the opposite of what I seek.   What would that mean in this context?  I might try setting up alerts for: 

     [ unanticipated benefits ] or
     [ unexpected good outcome ]

It's a trick, but one that's immensely useful.  

2. (Extra credit) Can you figure out a way to have this limited search run once / month?  (In this case, you'd probably want to have the search extend over the past month, not the entire year.)  

I'm going to save this part of the answer (it was extra credit!) for next week.  Look for my comments then.  Hint: this will center on Google's Programmable Search Engine (formerly the Custom Search Engine, CSE.) Programmable Search Engine and will show how to search for those small number of quality sources.  

SearchResearch Lessons 

1. There are a number of services that provide regular alert services.  Try them all!  You'll find that they cover very different information feeds and will give a broader coverage of ongoing news reports and research in your topic area.  

2. Once you've set up your alert feeds, prune them as necessary.  If you need to, set up a reminder to yourself to cut back on the alert feeds that aren't high quality.  Be sure to do this, or you'll end up polluting your personal information feed! 

3. To search for synonym phrases, look at other articles containing the phrases you use.  Often, these articles will have synonyms for the concept you seek. (Writers typically hate reusing a stock phrase over and over, so they look for other ways to say the same thing.)  

4. Another way to find synonymous phrases is to search on Wikipedia for the phrase you know, then read around finding other expressions.  That's the point of using the SITE: search. 

Search on! 

Wednesday, January 4, 2023

SearchResearch Challenge (1/4/23): How can I find latest updates on topics of interest?

 As you might remember... 

P/C MidJourney. Prompt: "unanticipated consequences, realistic"

... I'm also working on another book about Unanticipated Consequences.  In particular, you might remember that 18  months ago, I asked about how to do "slow research" on that topic.  

I'm happy to tell you that my research has been going well, although slowly.  The good news is that I'm getting close to the finish.  

But that brings up a different problem: I'd like my research to be as up-to-date as possible.  In particular, I'd like to be sure that I haven't missed anything in the past year.

This makes me think--how can people stay up-to-date on a particular topic?  The obvious way is to subscribe to blogs and newsletters that monitor the topic for you.  That's good, but suppose you'd like to get a bit more from the news directly?  

We've talked about using Google Alerts before, so that's not what I mean.  Let me frame this as an SRS Challenge for you: 

1. Can you find a way to limited search over a small number (say, 3 - 7) of high quality periodical sources of information for a particular topic for the past year?  (In my case, I want to search for articles on "unanticipated consequences" during 2022. Your topic of interest might be something different.)  How can I do that? 

2. (Extra credit) Can you figure out a way to have this limited search run once / month?  (In this case, you'd probably want to have the search extend over the past month, not the entire year.)  

This is the kind of tool that pro researchers use, or would like to, if they could figure it out.  Can you show them how to do this? 

Let us know how you found the answer to this! 

Search on! 

Sunday, January 1, 2023

2023 New Year's Thought: What does it mean to be curious?

In the pre-dawn quiet, somewhere on the north coast of California, I looked up into the soft, cold, gentle mist… 

Rainy day on the Northern California coast

... letting it play over my face.  It’s a reflective gesture, especially on New Year’s Eve when all of the year is worth rethinking.  Rain has fallen since the dawn of the Earth, and I’m just feeling the latest cycling of water from sea to sky to land back to sea.  But it puts me in a deeply contemplative state of mind.  What happened this year that was different from the year before?  How can I make next year even better?  What have I learned?

You might wonder about all of those questions, but as I lifted my face to the sky, I could feel each tiny raindrop strike my cheeks—a kind of constant spray where each pinpoint of rain registered—dot.. dot.. dot.. dot..—quietly making every point of my skin register a tiny scintilla of cold and wet.  You might think of this as a kind of celestial scanning of your inner self, the rain gods probing every square millimeter of your face and your life.  It felt a bit like video static on my skin.  

It felt like that image of William Gibson in Neuromancer, “The sky above the port was the color of television, tuned to a dead channel.”  I realize that there are people who have never seen the black and white video static of a dead television channel.  For you, here’s a YouTube video. https://www.youtube.com/watch?v=ubFq-wV3Eic (It’s a handy concept to have—the rapidly alternating pixels of black and white, rapidly flipping in a random order, flickering chaos. I don’t quite know how to convey that sense in words—video static as a phrase will have to do.)  

But the thing that popped into my mind was this: I feel as though the droplets are tiny and rapid and random and dense, but I also know the spatial resolution of touch on the skin of my face is pretty low. So how can I feel as though I’m in a shower of tiny video static droplets?  And while I’m thinking about that, I also start wondering how I can feel the cold of each droplet as it splashes down.  How fast are my cold receptors?  Can I actually FEEL each of those tiny droplets as a touch and as a chill, or is my perception a kind of synthesis of all the drops over a short period of time?  How much of my perception is accurate and how much is just made up by my sensory system?  

A psychologist would call this a question of veridical perception—that is, the direct perception of stimuli as they exist in the world.  Am I actually feeling what my perception actually indicates, or does my nervous system do a kind of quick extrapolation from the perceived reality to create the sense of video static on my skin?  

Naturally, I then spent a couple of hours doing a bit of research into the spatial resolution of touch. Doing the obvious queries on Google and reading widely in neuroscience and perception. I learned about the “two-point discrimination test” [1], and that we have different receptor neurons for heat and cold, yet again different receptors for touch and different ones for touch, different ones for stretching, and different ones for sensing vibrations.  Fascinating stuff, but in the end, I had to do my own two-point discrimination test on my face to determine what the spatial resolution is on MY face.  After using an incredibly sophisticated two-point testing device (that is, a paperclip I bent into parallel arms whose distance I could vary) I found that my cheek can tell touchpoints that are around 6 mm apart—around 0.25 inches.  This means that my face can only register drops that are 6 mm apart from each other.  Since my face is around 254 mm wide (assuming a circular face), I can really only feel around 42 different droplets from one side to the other. On the other hand, my lips can determine one- versus two-points with only a 1mm separation, so my face is at different spatial resolutions!  

But that’s not the way it feels.  I can't feel the difference in resolution--it's all of one piece.  I would have naively guessed that I could uniformly detect 100 drops of rain from cheek-to-cheek. The pattern of rain certainly feels that way as the rain washes across—it feels like a fairly dense spray of droplets—I would have thought I should feel at least 1000 droplets as the morning rain falls from heaven.  

And what about the cold? A little more research tells me that cold receptors are even farther apart, and much slower to react than touch. So, the sense I have of small cold drops hitting my face in a dense, random pattern can’t possibly be accurate. The rain is cold, certainly, but when the drop hits my skin, I register the drop’s strike, and the cold perception comes much later, probably right around the time another drop hits the same point.  

Once you start thinking about this, you also have to wonder—what is the composition of rain?  Are all drops the same size?  I noticed in the heavier rain that came later that there were also lots of little drops as well.  Are they shards from collisions of large raindrops, or did they never get merged into a big drop?  

Questions, always questions. 

This year I’ve spent a lot of time writing my SearchResearch posts—more time than I’d care to admit, but it’s worth it.  The process satisfies this deep inner itch I have--I want to understand the world.  

The nature of my curiosity is to constantly ask “Why?”  It’s an everpresent question that’s always on the periphery of my lived experience.  This year I walked through a park and thought “why is there an odd, flat space in the middle of the park?”  Or, seeing gnats swarming on the beach I wonder “Why do they do that?”  Seeing a horse in a nearby meadow leads me to recall from an adjacent memory about horses that while there are horse fossils in North America, and yet there were no horses here when the Europeans arrived:  Why?  

Curiosity is more than just asking random Why questions—it often builds off of something else you know.  In the park, there were no other flat places like that one—it just looked wrong: Why?  In the case of the beach gnats, I saw them all over the beach, but it wasn’t one giant beachball of swarming gnats, but several gnatty swarms in different places.  To ask the Why question about horses, I had to know that in 1492, there were no native horses in North America, and that there were horse fossils littering museums all over.  Why?  

Horse fossil from the La Brea tar pits in Los Angeles; Miocene.

Asking curious questions means noticing something about the world; it means knowing something is out of place, doesn’t fit, or doesn’t agree with another thing you know, or recognizing that you really don’t know how something works.  Then, ask that question, follow-up with a bit of looking.    

Fortunately, we live in a time where online resources let us do a bit of desk research and find the answers to these curious questions.  

What questions will you ask in the year ahead?  What role does innate curiosity play in your life?

-- Curiously yours 

-- Dan 


1. That is, how far apart do two points have to be on your skin before you feel them as two points rather than one? Answer: it varies widely, from 1 mm at the fingertip to 40 mm on the skin of your thigh).  I also learned that a “receptive field” is the region that’s sensed by a touch receptor.  For more details:  http://www.scholarpedia.org/article/Receptive_field