Taking a fortnight break

 

Sunrise over Palo Alto, CA * Nov 27, 2024

It's time for a little Rest & Relaxation for those of us laboring on the SearchResearch Rancho.  

I'm taking the next two weeks off to go on one of those trips that you've been planning for a long time... one that is finally becoming a reality.  I'll share some details after I get back, but in the meantime, SRS will be quiet until mid-December when we'll pick up the Challenges one more time.  

Hope you all have a restful and relaxing couple of weeks while I'm wandering around the planet, looking for ever more exciting SearchResearch Challenges.  For US folks:  Happy Thanksgiving!  

Best wishes for a peaceful first few weeks of December.  See you back here around December 18.  

As always, Keep Searching... it's a big and curious world out there. 

-- Dan 

P.S.  Below:  The way I imagine the SearchResearch Rancho, heavily influenced by the old Californio Ranchos... Thanks to Gemini for this vision of the [SearchResearch Californio Rancho in plein air style].  I assume there's WiFi here... 

Answer: How to look at more trend data?

Little tricks and techniques to know... 

Scales to get relative and absolute weights.
P/C Jean Poussin, from Wikipedia. (CC BY-SA 3.0)

As I mentioned last week, sometimes the tools just don't quite let you do the analysis you'd like to do. 

The key question was how to create a chart like this, with more than five variables with data from Google Trends?  (The built-in graphing tool of Google Trends only allows you to compare 5 different queries!)  

As you can see, here I'm showing 6 different holidays on the same chart. What if I wanted to compute 8 different Google Trends search volumes.  (Here the chart shows 6 different US holiday search volumes.)  

So... the obvious Challenge is this: 

1.  How can you get more than 5 search volumes compared side-by-side?  

As you probably noticed, Google Trends has a lovely built-in graphing tool... that's limited to 5 trend lines. 

Fortunately, there's a way to export the CSV of the data and that will let you compare everything side-by-side.  

If you look at this Google Trends display for searches of cranberries, oranges notice the Download symbol just to the left of the angle brackets ( <> ) to the upper right of the chart.  Clicking on that will download a CSV of the data.  If you've selected multiple Trends, then you'll get a CSV.  (And notice the lovely seasonality effects of cranberries and oranges.)   

Here's what that CSV looks like in Google Sheets: 

The thing we have to figure out is how to get 6 (or more) variables into our chart. 

Suppose we want to understand the search relationships between cranberries oranges walnuts figs blueberries.  Here's that chat on Google Trends for the past 5 years.  

Now, to add in another seasonal fruit, we have to figure out which fruit scores 100 somewhere in this chart.  If you look carefully, it's a toss-up between blueberries and figs.  BUT.. by scrubbing your mouse over the data (or looking at the CSV) you can find that the high point is figs for the period Aug 25-31, 2024, so figs will become our common baseline.  Every Trends analysis we run has to include figs so they will be comparable.   

What we'll do is to run another Trends analysis, but keeping figs in the set so we have a consistent basis to compare to.  So now, let's add in raspberries.  Here's what I get: 

It's fairly simple to then get the CSV for this data set.  Here I'm looking at it in Google Sheets: 

Then copy-paste the raspberries column from one spreadsheet to the other, and use the built-in graphing ability of the spreadsheet to give you what you want.  

That's a complicated chart, but notice the seasonal effects of cranberries and oranges.  Figs have a seasonality, but it's more complicated.  (I leave that to you to figure out why!)  

One thing to notice: If you add in a fruit that has values larger than the fig numbers, you'll have to choose THAT fruit as your baseline.  Figure out ahead of time which numbers are the largest so you can use that one as your basis of comparison. 

 

2. Since these are all relative values, are these BIG numbers, or relatively small ones?  How would you know?  

This is a ongoing puzzle for all Trends users--the numbers are all percentages with respect to each other.  That is, of the six fruits shown above, the values are all percentages of the maximum value.  (That is, 100% is figs for the period Aug 25-31, 2024.)  So, the value of 25 for blueberries on March 8, 2020, means that the "Relative Search Volume" (RSV) for blueberries on that day was 25% of the volume for figs on Aug 25.  All of the numbers are RSV values for each of the search queries.  

So, how do you get a sense of what's big or small? 

The best way to do this is to compare the RSVs to something you already know about AND is pretty flat.  

Here's a Trends chart for raspberries, blueberries and aspirin.  I used aspirin because it's fairly constant throughout the year.  Everyone has headaches all the time...   

Depending on what you're trying to analyze, you might have to find other fairly flat queries (try pharmacy or time as other queries that are fairly unchanging).  

By comparing your "fruit query RSVs" to other queries that you know about (and are fairly flat by comparison) you can get a sense of (but not the exact values for) the size and sweep of changes over time.  

SearchResearch Lessons 

1. When the tool you're using doesn't give you what you need, consider exporting the data and using a tool that DOES work for your analysis.  That's the big lesson here.  Trends won't give you more than 5, but other tools do.  Export and align the data to see what you want. 

2. Remember to compare apples to apples.  Or, in Trends case, remember that everything is an RSV (i.e., a percentage of the max).  Do NOT create two non-comparable data sets--you have to have one column in common--the value that has the max value for your data.  

Hope some of you find this useful when doing your own work.  

Keep Searching! 

SearchResearch Challenge (11/20/24): How to look at more trend data?

 Limits in your searching can be so annoying... 

Scales to get relative and absolute weights.
P/C Jean Poussin, from Wikipedia. (CC BY-SA 3.0)

... and there are times when you'd like to look at the data, but the tools just don't quite let you do the analysis you'd like to do. 

This week, I happened to be looking at which of the most common holidays in the US created the most search traffic.  Why? As you know, many people look to Google search traffic as an approximation for the level of interest in a topic, and I wanted to know which of the holidays occupies more mental space in the brains of US citizens?  Christmas?  Thanksgiving?  The Super Bowl?  Or.. what?  

To figure this out, I really needed to compare multiple search queries at the same time.  Sounds great.  

But there's a problem: How can I create a chart like this, with more than 5 variables?  (Google Trends only allows you to compare 5 different queries!)  

As you can see, here I'm showing 6 different holidays on the same chart.  This is a "Relative Search Volume" chart, and not the absolute numbers.  When Google computes such charts, the measure with the largest volume will be normalized to 100%.  All of the other data is based off of that. 

So... the obvious Challenge is this: 

1.  How can you get more than 5 search volumes compared side-by-side?  

And a corresponding question: 

2. Since these are all relative values, are these BIG numbers, or relatively small ones?  How would you know?  

Tell us what you think the answers to the Challenge are in the comments.  What clever method did you use to get to 6, 7, or more variables on the same Google Trends chart?  

Keep Searching! 

Answer: How long did they last?

Finding the end of something is hard. 

P/C David McEachan, from Pexels.com

P/C Bera Yannick, from Pexels.com showing typical offerings--bread, beer, animal leg

Pyramids and all of the funeral gear created by the ancient Egyptians was clearly intended to last forever.  (They said as much in their tomb writings.)  

Part of their religion included leaving offerings of food, drink, clothing, and ointments to their dearly departed.  This would ideally happen often, maybe even daily, but certainly on special festival days.  

But... the practice of delivering offerings to these places did not last forever.  But how long did they last?  That's this week's Challenge: 

1. After the Pharaoh's death, how many years did people keep bringing offerings to the pyramids?  Was it just a few years and then they gave up?  How long did the offerings keep coming?

Finding the ending of something that was supposed to last forever can be a bit tricky.  I wasn't expecting to find a memo (or a papyrus or ostracon) with a memo saying "Wajit just dropped off the last offering at the tomb of Wepwawet on this date, XXXX BCE."  

So how can you approach a question like this?  

I tried the obvious LLMs question: 

[How long did ancient Egyptian funerary cults last?  That is, after the death of a pharaoh, a cult would be set up to provide offerings to the dead king.  How many years after the pharaoh’s death would the cult keep bringing food to the funeral site?] 

The answer I got from ChatGPT was: 

Ancient Egyptian funerary cults established to honor deceased pharaohs often lasted for centuries, though their duration varied significantly based on the era and the prominence of the pharaoh. In the Old and Middle Kingdoms, particularly, these cults could persist for several generations or even longer. The state-supported cults of powerful pharaohs, such as those of the great pyramid builders (e.g., Khufu, Khafre, Menkaure), were maintained for a few hundred years with dedicated priests offering food and performing rituals at mortuary temples.

However, over time, the continuity of these cults would depend on the resources allocated by subsequent rulers and the changing religious and political climate. By the end of the New Kingdom, many of the older mortuary cults diminished or ceased entirely as political instability and economic challenges led to a reduction in resources and attention given to maintaining such practices.

Thus, while the initial dedication and rituals following a pharaoh's death were intense and sustained, the active period could range from a few generations to potentially several centuries before gradually being reduced to nominal or symbolic remembrance.

Interesting!  To summarize: the funerary cults would service the temple with offerings from "several centuries" or "several generations," with significant variance.  

I then asked for a few references to support these claims.  I got these suggestions: 

1. "The Complete Gods and Goddesses of Ancient Egypt" by Richard H. Wilkinson 

2. "Death and Salvation in Ancient Egypt" by Jan Assmann 

3. "The Oxford History of Ancient Egypt" edited by Ian Shaw 

This a pretty reasonable set of books.  Modern Egyptology has been a field of inquiry since the 18th century, when Napoleon and his army went on an invasion of Egypt with 160 "savants" (scholars and academics), bringing back loads of materials for study, especially about Ancient Egypt.  While scholarly opinion will shift with time as we learn more about the time, I had figured that there would be substantial writings about the funeral practices, and perhaps one of them would mention when the offerings stopped.  These three books were a great starting place.  

But how to get access?  The first two books I was able to check out of my local library "Complete Gods" was in the stacks, while I had to order "Death and Salvation" by interlibrary loan.  But that worked pretty well and I had it two days later. The Oxford History of Ancient Egypt is available in preview mode on Google Books. 

When looking these books up in the catalog, I also noticed “Death and the Afterlife in Ancient Egypt”  by John H. Taylor--also really relevant.  

Even better:  "Death and Salvation," "Death and the Afterlife" and "Oxford History" are available on Archive.org (the Internet Library).  (You have to login to be able to borrow and read these books, but that's pretty simple.)  Once there, you can search within the book.  Also wonderful, once you're looking up one of these books, you'll see other closely related books. Here's the UI from Archive.org: 

A set of books that are "near" to "Death and the Afterlife in Ancient Egypt" at Archive.org.  Very much like browsing the shelf in the physical library, but one with a very deep collection. 

So now, with these four resources, I started searching and reading each section that mentioned "cult" or "offering" or "generation."  ("cult" because I wanted to read everything about the funerary and mortuary cults; "offering" because I wanted to read how "offerings" were taken to the tombs and for how long; "generation" because I was hoping to find something like "... and people kept bringing offerings for several generations..."  

After a happy couple of hours reading probably 200 pages of all these texts, here's what I found:  in general, the offerings were kept up for 1 or two generations after the death of the tomb-owner.  (It seems very doubtful that they lasted for "several generations" as ChatGPT said.) However, read on...  

Suggested offerings for the care and maintenance of the dead, including head of cow, goose, and leg of gazelle.

Since the role of the offerings were to keep the spirit of the dead well-supplied with food and essentials for the afterlife,  there was a backup strategy:  

Death and the Afterlife: 

Page 22: "Since the [deceased] name was so closely linked with the prosperity of the bearer, survival of death was linked to remembrance of the name. It was necessary for it to be pronounced in the context of the offering ritual (see pp. 94 and 192-3), in order to provide nourishment for the dead. Failing that, funerary texts appealed to anyone who might visit the tomb in future years to pronounce the appropriate formula so as to supply offerings for the dead person; thus, as long as the name was remembered, the dead would continue to live." 

And therefore, the role of the offerings would be easy to continue over the years.  Can't make it with fresh bread and beer?  Just have a visitor pronounce the names of the offering goods, and that'll work for the dead. 

But how long did the original offering-bearers keep up that tradition of bringing goods to the funeral site?   

Death and Salvation: 

“For an Egyptian, his hometown was the place not only of his own tomb but also of the tombs for which he, as descendant, was responsible. The meaning of this concept of burial in one’s place of origin was that in this way, the deceased did not fall out of the “connective,” life-giving structure of affiliation but rather remained included in his community. This inclusion took many forms. The most important, but also the briefest and least certain form was the organized mortuary cult; it seldom lasted more than one or two generations..."  

Death and the Afterlife:

Page 41: 

Relatives bore the chief responsibility for maintaining the cult of the dead, a duty which fell above all to the eldest son. Inheritance of family property was linked to this act, but in any case providing for dead parents was regarded as an important filial duty (see p. 171). The dead, indeed, exercised considerable influence over the economy of Egypt, since substantial resources, manpower and material wealth were expended in equipping them for eternity. 

Page 44: 

The Egyptians were fully aware that even the most careful preparations for death could not avert the eventual decay or destruction of the tomb and the cessation of the mortuary cult. The positive, hopeful tone of the mortuary texts is balanced by a genre of literary compositions, mainly dating to the Middle Kingdom, which express skepticism about the worth of making elaborate preparation for the afterlife. This theme was taken up in the Harper's songs, the most famous of which is stated to have been inscribed in the tomb of one of the kings named Intef at Thebes. The general tone of these texts is that tombs fall into ruins, mortuary cults do not endure, and that no one returns from the realm of the dead and strengthen the hopes of the living.  Therefore, one should live only for the present, enjoying the pleasures of life to the full…. 

Page 95: 

Throughout most of the pharaonic period the sustenance of the dead was taken care of by magical means. Funerary texts indicate that it was not the physical body but the ka which required feeding (see Chapter 1) (see fig. 56). The ka was dependent on the world of the living to make this possible. This need was met chiefly through the establishment of a mortuary cult, served by relatives of the deceased or by priests, whose duty it was to present offerings to the dead in the context of a formal ritual. This took place in the tomb chapel, to which the ka ascended from the burial chamber below (see Chapter 5), passing through the false-door and taking up temporary residence within the statue of the deceased in order to receive its sustenance. In terms of its procedures, the offering ritual was closely related to the magical purification and provisioning of the images of the gods which took place every day in the great cult temples. The ka of the deceased, like the divinity embodied within the cult-statue in the temple, consumed only the essence of the foodstuffs placed on the offering table. The offerings then ‘reverted’, or, in other words, were eaten by the priests or by those who had performed the ritual. The underlying notion that physical digestion did not take place removed the obligation to ‘neutralise’ (through offering them to the dead) much-needed provisions which might otherwise be consumed by the living...

Mortuary cults were established with great care to ensure that the funerary rituals would continue to be performed from one generation to the next (see Chapter 5). However, despite the most elaborate precautions, it was the fate of all such cults ultimately to fall into abeyance. Though the spirits of long-dead kings might continue to be maintained through temple cults, with the support or consent of the reigning pharaoh, for non-royal individuals, cults of remote ancestors (even grandparents) were probably rarely maintained for more than a generation or two. The Egyptians were well aware of the futility of trusting to tomb and cult for eternal survival, as literary compositions often emphasized. (see p. 45).

So the Egyptians understood that things don't keep running forever, even if that was the religious goal.  

But more, I hadn't thought about the differences between "official" (that is, state-run) funeral cults and the ability of the government to continue to provide offerings versus smaller, personal, family-run tombs. Of course, the families would maintain the schedule of grave offerings, but as suggested here, that typically runs out in a generation or two.  

But did STATE-run offerings last longer?  And then there's the funerary cult of an individual (e.g., Pharoah Ramesses II) or a powerful leader's extended family?  (e.g., Ramesses II 52 sons and many, many daughters and many wives, with a large mortuary complex)  

In a surprising connection, as I was reading about the Ramesses II funerary cult, I did a search for [ Ramesses II funerary cult ] and discovered several articles about the recent discovery of thousands of mummified rams heads at the Ramesses II temple in Abydos.  (Archaeology Wiki, Smithsonian) The interesting thing here is that offerings were being made to the temple of Ramesses II in Abydos a thousand years after his death.  

It's a bit like what we see in our own lives--individual gravesites are typically maintained by families for a generation or two, then people move on and the sites are abandoned to the vagaries of time.  But official, state-run memorials get preserved for the length of time of the official organization, which can be centuries.  

An example of a modern cemetery with offerings. The gravesite of L. v. Beethoven, with flowers and gifts from current acolytes. P/C by Dan, Central Vienna Friedhof, Austria.

2. A related question: How were the people who were the bringers-of-offerings organized?  Was there a central church organization or division of the government that handled such long-lasting affairs?  What were these organizations called?

I didn't have to do this search explicitly since the answers were scattered throughout the texts I read.  For smaller funeral sites, the offerings were done by the familes for a generation or two.  For larger (i.e., more important people), the work was done by a fund that supported the maintenance... until it didn't.  For VERY important people, the offerings continued for many, many years, supposed by the state itself.  

Unless, of course, the next regime took an active dislike to you and tried hard to erase all trace of your existence, a fate literally worse than death for the Egyptians.  Around 1350 BC, Pharaoh Amenhotep IV decided that all the gods of Ancient Egypt were a lie, except for one: the sun God Aten. He build a new capital for him in the desert 200 miles south of Cairo, and changed his name to Pharaoh Akhenaten.  The campaign of damnatio memoriae against Akhenaten and his successors was begun by a later pharaoh, Horemheb, who decided to erase from history all pharaohs associated with the unpopular Amarna Period, including, and especially Akhenaten.  This happened more often than you'd believe. The pharaoh would be erased, and all of the mortuary cults that were dedicated to feeding him in the afterlife were, well, killed off.   

Akhenaten, Nefertiti, and children from the time before Horemheb.  

Akhenaten literally erased from history.

SearchResearch Lessons

1. Books are still incredibly valuable as resources!  Don't underestimate the value of your local library. You might get inspiration from your AI model, but checking with the books to get the whole story (and the context!) is incredibly handy. 

2. Online books are better in some ways.  While the index in the back of a book is a labor of love, full-text indexing can show you things that you literally cannot find via the index.  Learn how to use the various online book resources (e.g., Internet Archive's online texts; Google Books; Hathi Trust; various e-book systems, etc) that provide word-level searching, and then skim through the book text searching for the sections that talk about your topic of interest.   

3. Don't underestimate the value of finding by accident.  The shelf-like view of Archive.org's "closed related" texts is pretty great.  Also keep track of things you run across in the course of your research.  (I'll say it again: learning to keep notes as you work is incredibly valuable.)  

Keep searching! 

SearchResearch Challenge (11/6/24): How long did they last?

Traditions die hard. 

P/C David McEachan, from Pexels.com

P/C Bera Yannick, from Pexels.com

But it's clear that some traditions can last a long, long, long time.  But how long?

I remember reading that ancient Egyptians would bring offerings of food, drink, clothing, and ointments to their departed leaders (as seen above). This ritual would happen daily when priests would attend to the tomb and make sure that those who have passed on would have sufficient food and goods to make it through the afterlife. 

Since pyramids (and smaller mastabas) were intended to last forever, one has to wonder, how long did people keep bringing offerings to the tombs?  

As we know, the "forever" thing didn't quite work out.  The Great Pyramid of Giza was built as the tomb of pharaoh Khufu, who ruled during the Fourth Dynasty of the Old Kingdom. Built around 2600 BCE over a period of about 26 years, it was apparently plundered by 1570 BCE.  That leaves the obvious question--were priests and family members STILL making offerings? 

Wondering about this leads to today's Challenge: 

1. After the Pharaoh's death, how many years did people keep bringing offerings to the pyramids?  Was it just a few years and then they gave up?  How long did the offerings keep coming?

2. A related question: How were the people who were the bringers-of-offerings organized?  Was there a central church organization or division of the government that handled such long-lasting affairs?  What were these organizations called?

I'll show you my work next week, but in the meantime, do let us know how you did your research!  We all want to learn the methods you used to get to the answer.  

Keep searching!  

Answer: How is it made? (flannel and snake oil)

 Flannel up! 

Image by Gemini. [young woman wrapped in flannel blanket at window, pre-Raphaelite]

We've had a string of chilly mornings, perfect for answering a couple of SRS Challenges. Get that cup of tea and start searching!   

But back to flannel... how?  

1. Can you find out how flannel cloth (e.g., that used for comfy sheets or shirts) is actually made?  How is that distinctive nap made?  

Quick answer:  it's brushed by a mechanical device that raises the nap of the underlying cloth.  

A quick: 

     [ how is flannel made ] 

teaches us that "Flannel may be brushed to create extra softness or remain unbrushed. Brushing is a mechanical process wherein a fine metal brush rubs the fabric to raise fine fibres from the loosely spun yarns to form a nap on one or both sides. If the flannel is not napped, it gains its softness through the loosely spun yarn in its woven form."  Wikipedia on Flannel 

Reading that made me realize that the word napped suggested that there was probably a process for doing the napping.  So my next query was: 

     [ process of raising the nap on cloth ] 

which took me to the "Napping of Cotton Fabrics" technical document, which goes into great detail about how flannel is actually made.  In there, they point out that: 

NAPPING - A mechanical finishing process in which fibers are raised on the surface of a fabric by means of teasels or, on contemporary equipment, by rollers covered with steel napper wires. Other names for napping are Gigging, Genapping, Teaseling, and Raising.

That makes sense.  Of course, once you've napped the fabric, you need to trim it back down, so that means: 

SHEARING - This process normally follows napping, and involves the mechanical cutting or trimming of fibers projecting from the surface of the cloth to produce a uniform and level pile.

They also tell us all the details: 

"Different napping results can be obtained by varying the speed of the napper rolls relative to the speed of the fabric. A positive napping action occurs when the points of the napper wire travel at a speed greater than the speed of the fabric. Additionally, the pile versus counter pile roll speeds can be varied. In general, greater pile energy produces a loftier nap, while greater counter pile energy produces a shorter, thicker nap. When the napper roll wires travel at a speed slower than the speed of the fabric, a tucking or felting effect of the fabric pile occurs. When a napped fabric is felted, the raised fiber ends are tucked back into the base of the fabric, producing a smoother fabric with better appearance retention after washing." 

A double-action napper device made by Gessner.

Of course, I wanted to see this in action, so a jump over to YouTube and a search for: 

     [ machine making flannel cloth ] 

led to a bunch of videos... many of which look good, but you have to watch them with care to find a segment that actually shows the napping (or "brushing") process.  Here's one that I found (link to video).  

At the 8 second mark, you can see the red fabric running through the rollers with the napping wires.  

Interestingly, Regular Reader Arthur Weiss got much the same results by using Perplexity (see his comment in the original post).  The LLM also gave a few additional pages that I didn't see in my regular Google search, so there is a breadth and diversity of results here, which is good to see.   

And since multiple web pages mentioned teasels as the original tool used to make flannel, I did a search for: 

     [ teasels used to make flannel ] 

and found this device: 

A "teasel cross" National Trust Inventory Number 117317.14On display at Lavenham, Suffolk

.

Here's a closeup of what the teasel seed head looks like: 

Teasel seed head closeup.  A few of the seeds are geminating after recent rains. p/c Wikimedia

2. Is snake-oil a real thing, or is the falsity of "snake oil" that it doesn't really exist?  How much fat is on a snake that could then be rendered into oil?

Snake oil has come to mean something fake, often a medicine that's passed off as something efficacious, but actually has no real merit.  

The Wikipedia article is interesting, telling us that there really IS a kind of snake oil that's traditionally made from Chinese water snakes.  

But in the US, there was a famous case that in 1917, when Stanley's Snake Oil was discovered to contain no actual snake oil, creating the notion that bottles of snake oil (and their salesmen) were essentially fraudsters.  

To our question, though, how much fat can a snake have that might then be rendered into oil?  

A quick query for: 

     [ fat on snakes ] 

leads to any number of discussions of how to deal with your obese pet snake.  Curious about how and where fat is stored on a snake, I did a follow-up query: 

     [ fat snake dissection ] 

and found a large number of dissected snakes showing exactly where the fat stores are kept.  (Often in the tail, but fat nodules can be found anywhere!)  It's remarkable how many images there are online.

So, much to my surprise, snakes CAN be fat.  It's not incredibly common in the wild, but it clearly does happen when a snake lives in a meal-rich environment.  

How about the utility of real snake oil?  

     [ snake oil effective ] 

led me to a large number of articles about the legacy of snake oil salesmen, but also to a number of discussions of the amount of Chinese snake oil may have had.  There seem to have been real benefits due to its high concentration of the omega−3 fatty acid eicosapentaenoic acid (EPA)—more than that of salmon

So... I learned a bunch here.  Real snake oil can be useful, but it's nearly impossible to buy actual snake oil these days (and apparently even more difficult in the past).  

SearchResearch Lessons 

1. Finding a video of the actual operation of "how it's made" devices can be difficult.  There is not, as of yet, a great video search system.  I found LOTS of "how flannel is made" videos that didn't actually show you anything of interest.  At this point, dedication is still required.  Hang in there. 

2. Using an LLM as a "lead generator" is a good idea.  It's yet another way to broaden the set of leads you follow when pursuing a particular topic.  

Keep searching.  

SearchResearch Challenge (10/23/24): How is it made?

 It's that time of year... 

Image by Gemini. [young woman wrapped in flannel blanket at window, pre-Raphaelite]

... "thou mayst in me behold / when yellow leaves, or none, or few, do hang..."  

Or, less poetically, the time of year when the flannel comes out.  

As I put on my beloved flannel overshirt on this chilly morning, I got to thinking about the fabric.  While I have a pretty good idea about how most fabric is made, I really don't know the mechanism of flannel.  It's warm and comfy, but how do they DO that?  How do they make it so soft and warm?  

More generally, how things are made is always a great SRS topic.  As a flute player, I've wondered how flutes got built.  There are several great YouTube videos showing the entire construction process.  (Here's an older video showing all of the steps.)  Fascinating--and now I know.  

But back to flannel... how?  

1. Can you find out how flannel cloth (e.g., that used for comfy sheets or shirts) is actually made?  How is that distinctive nap made?  

To continue the theme of how-things-are-made, the other day I happened to see a snake zipping down a forest trail in front of me.  A bit later, I saw a reference to someone as a "snake oil salesman."  I understand what that means, but every snake I've ever seen has not a gram of fat on it.  So... how? 

2. Is snake-oil a real thing, or is the falsity of "snake oil" that it doesn't really exist?  How much fat is on a snake that could then be rendered into oil?

When you search these things out, be sure to track HOW you found your answers and let us all learn how you did it.  

Still wondering about flannel creation... 

Keep searching.  

Answer: Finding similar content?

 Finding something similar... 

P/C Gemini. [create an image of a hall of mirrors with trees and mountains]

... but not exactly the same is a common SearchResearch thing to do.  As I'm doing my own research work, I'll want to find another take on a topic, and would really like to know how to find a similar / closely-related page or site.    

This common research step is our Challenge for this week: 

1. How can you find related sites (or content) to something that you know about?  Suppose you want to learn more about Search and Research methods:  Can you find similar or related websites to our trusty SearchResearch1.blogspot.com?  

Last week I gave you a BIG HINT:  If you do the obvious search e.g. [how to find related sites on Google], it will tell you that you should use the related: operator has been deprecated since June, 2023.  Don't believe it: It is no more--it has ceased to be. (See the note from Google's SearchLiaison.)  

Why is it so mistaken?  A: There are thousands of pages that discuss how to use the related: operator... but only a very small number that tell you that it no longer works.  Unfortunately, both search engines and LLMs work with the vast majority rather than actual bits of knowledge.  You could say that the site: operator no longer works (don't worry--it still does!), but it would take the LLMs just about forever to figure this out. 

Alas, Google currently tells you when you ask [how to search for related pages on Google]  

WRONG!
Searching for how to find related sites on Google.  It gives you a bunch of bogus information.  If I were a linguist, I'd put a giant asterisk next to this set of results.

What IS a reasonable "find related" strategy?  

I asked a couple of LLMs "how to find sites that are similar to X," and while a lot of the advice wasn't great, I DID extract a few gems.  

SimilarSites.com - doesn't work for our SearchResearch1.blogspot.com site (but see below--it DOES work for 3blue1brown... bear in mind -- your mileage may vary).  

SimilarWeb.com - DOES work for SRS, but the similar sites are mostly math or education sites (e.g., 11011110.github.io or blog.discoveryeducation.com).  Those are fine, but not quite on topic.  

If you're a Regular Reader, you know that what our SearchResearch site is REALLY all about is how to be a better searcher.  We cover a lot of other ground in our quest to improve online research skills, but at heart, we're a team that knows about how to do online search.  

But the blog is a long series of Challenges followed by Answers. What is it about?  

To get the gist of SRS, I took a few posts (in particular, the last 8--four Challenges and four Answers), copy/pasted them into a single file, uploaded it to Gemini and asked it: 

This file is a sampler of several pages from the searchresearch1.blogspot.com web site.  Can you summarize these pages and tell me what this website is all about? 

It's not a perfect summary, but it's not bad.  Here are a couple of excerpts:  

This website, searchresearch1.blogspot.com, is a blog dedicated to exploring the intricacies of online search and research methodologies. It delves into the art of utilizing search engines effectively, with a particular focus on Google, and provides valuable insights into how to conduct research efficiently and intelligently.

The blog's author, Dan Russell, frequently presents "SearchResearch Challenges," thought-provoking questions designed to test and enhance readers' search skills. These challenges range from identifying the oldest city in the Americas to finding related websites with similar content.

A recurring theme throughout the blog is the concept of "unanticipated consequences," highlighting the unexpected outcomes that can arise from research and the importance of being prepared for surprises. The author emphasizes the need for persistence in research, advocating for continuous learning and exploration.

Overall, this website serves as a valuable resource for anyone looking to improve their online search and research capabilities. It encourages critical thinking, effective search strategies, and a deeper understanding of the complexities involved in finding and evaluating information online.

But from this summary, we could extract a couple of decent search queries to find related sites.  I extracted two: 

     [ art of utilizing search engines effectively,
       valuable insights into how to conduct research
       efficiently and intelligently ] 

     [ anyone looking to improve their online search
       and research capabilities. It encourages critical
       thinking, effective search strategies, and a deeper
       understanding of the complexities involved in 
       finding and evaluating information online.] 

Yes, I know those are very long queries, but the old advice has to be updated: One used to look for short, precise queries that would get you just a few, highly targeted results.  But THIS Challenge is asking a broad question that has multiple aspects to it.  For questions like this, longer queries actually work very well.  Here's some of the results:  

Note that not all of the query terms are in each result.  Doesn't matter. 
We're looking for related sites, not exact duplicates.

These two queries led to quite a nice collection of sites that are closely related to SRS. Here's a short list of interestingly related sites that I didn't know about!   

Zampila (on research methods), UniversalClass (on how to do online searches), Marcos Esteve blog on data science and search... There are a bunch!  

2. Same question, but about a different website:  Again, one of my favorite sites, www.3blue1brown.com (an excellent science/math/tech site with lots of exquisite videos illustrating complex topics).  

SimilarSites.com works for 3blue1brown, and points to sites like TomRocksMath.com and BetterExplained.com -- those are great sites that ARE very similar to 3blue1brown.  

Doing our "summarize the website" works less well for 3blue1brown since it's primarily a video site, but you can still patch together a set of writings (such as the "About" page for the site) and ask for another summary, then use that for your search.  When I did that, it led to me to: 

Mathologer: This YouTube channel delves into fascinating math puzzles and concepts with a focus on visual proofs and surprising connections.

Numberphile: Another YouTube channel, Numberphile explores a wide range of mathematical ideas through interviews with experts and engaging visuals.

Desmos: This free online graphing calculator allows you to visualize functions, explore geometric concepts, and create interactive mathematical models. It's a fantastic tool for hands-on learning.

GeoGebra: Similar to Desmos, GeoGebra offers powerful tools for visualizing geometry, algebra, calculus, and statistics. It's great for dynamic exploration and creating interactive simulations.

Overall, this approach works rather well.  

In the comments, remmij reminds us that for any question, there is a pretty decent chance that there's a Subreddit about.  In this case, searching on Reddit for "3blue1brown" quickly leads to "Looking for more channels like 3blue1brown" and a lot of suggestions for similar sites.  

Mott Given wrote in to point out https://explore2.marginalia.nu/ -- a site that finds nearly related sites.  

Regular Reader Krossbow pointed out the Yogurrt "discovery engine" (which didn't work so well for me, but YMMV).  

There are probably more such tools out there.  Let me know and I'll add them to this page.  

SearchResearch Lessons 

1. Be wary of the specific details about how to use software given to you by search engines or LLMs.  They're often out-of-date--partly because operations change quickly, and partly because there might be millions of citations about how something works, but only ONE reference to that function being deprecated.  Unfortunately, that's not the way LLMs work--you can't just say "this doesn't work anymore" and have it "learn" that!  

2. There are tools worth knowing: SimilarSites.com and SimilarWeb.com both give decent results. 

3. Creating a longer summary description of the site and asking your search engine to do a search on that description often gets you into the neighborhood. As I mentioned, that was a crazy method to do a couple of years back, but these days, it often works just fine.  

4. Don't forget to look specifically for recommendations for "more X like Y"--you never know when Reddit will have exactly the thing you seek.  

5. Ask your friends...  That's how we found a few additional "related" tools--friends comments left here!  

 

Keep searching.