Friday, October 22, 2021

SearchResearch Challenge (10/22/21): Are these documentaries difficult to find?

 Documentary and older films are a
   quiet pleasure... 

... I love watching them, but sometimes find them a bit difficult to find.  Maybe you have this problem as well.  

Ideally, I'd like to watch them for free, but I'm totally happy to pay for the privilege of watching, if I can find a reasonable price.  

Usually, when I see a reference to a documentary I want to watch, I jot the name down on a slip of paper.  Then, a few days, weeks (or years!) later, I try to find them online.  

Sometimes I admit that my writing isn't as careful as it should be, so I end up having to search for films whose names might not be exactly right.  

But this is something we all face--finding something that's relatively obscure, and perhaps with a little error that crept in along the way.  

Can you find these films so I can watch them online?  


1. Garlic is as Good as Ten Mothers.  I know this is by Les Blank, it's about garlic, and maybe it's about motherhood, but that's all I know.  

2. Metropolis.  Not a documentary, but a super famous German expressionist film from 1927.  Can you find it online? 

3. The Disappearing Cape Breton Violinist.  My handwriting was shaky on this one, so the title might be a little off.  I'd still like to watch it, though.  Can you find a link to an online video of this? 


Note that I don't want to see the trailers for these movies, but the actual full-length production.  

As always, I'm interested in HOW you found these videos of these.  Tell us so we can all learn!  


Are there videos that you'd like to see that YOU find difficult to locate?  Bring them up here, perhaps the crack SRS team can help you out! 


Search on! 


P.S.  The math analysis of the shadow-date-time problem will be my next post early next week.  


Wednesday, October 20, 2021

Answer (3): Can you get the time and date from a shadow?

 Can you actually get the time and date from just a shadow?  (installment #3) 


This is part 3 of the time & date from a shadow Challenge. This is all about using tools, and checking to make sure the tools work the way you think!   

-----------


We're back trying to figure out when this satellite photo was taken.  



1.  What day and time-of-day was this photograph taken?  

In our last post, I tried using a lot of photographic resources, but didn't get very far.  Turns out that while the Skytree has a LOT of photos, very few are the ones I need!  


But I found that by using SunCalc in an interactive mode, I could move the time slider at the top of the page (that's the sun image at the top).  You can check it out here.  In particular, if you dial in Jan 29, 2020 at 10:42AM, you'll see a modeled shadow that looks just like the photo above! 


Using Suncalc, I can find a time/date that shows the shadow just over the Sumida bridge!  
I just read the time/date off the panel on the left side.  10:42AM, Jan 29, 2020


(And with even more playing around, you'll quickly realize that the shadow is ALSO the same position on Nov 12, at 10:14AM.  More on this later.)  

But now I'm curious: Is THIS model correct?  

I did a search to see if I could find another shadow-casting / modeling online app, and I found a few.  The one that appealed to me the most is Shadowmap, which, if you play around with it, can give you a very similar image (note that the underlying base map of Shadowmap doesn't include the Sumida bridge, so I had to guesstimate the position.  Luckily, it's pretty close to what SunCalc says... 

....although Shadowmap has the time as 11:12AM on Jan 29, 2020.  

By using ShadowMap in the same way, I could find an identical map model... but for a slightly different time.  Here, the time is shown as 11:12AM on Jan 29, 2020.  Why is it 30 minutes different than SunCalc? 

So... the two models are very, very close... 

   SunCalc            10:42 AM 
   Shadowmap          11:12 AM 

but 30 minutes difference is annoying.  Which one is closer to reality?  How can I check?  

I know.  I can go out and measure a real building, take a photo of a real shadow, and then see what the predictions are!  

Early Sunday morning, I went off to a nearby building and took a photo of its shadow at 9:03AM on Oct 17, 2021.  Here's what I found--the shadow of the building goes out into the street almost to the center--it's about 1m short of the center.  


(Side note:  Turns out that it's hard to get clean photos of Silicon Valley building shadows.  They tend to be surrounded by trees, which makes getting the edge of the shadow somewhat difficult.  This is the best I could do on Sunday morning!) 

Here's what Shadowmap predicts as the building shadow for this time (the green triangle shows where I'm standing, looking down the street to the south).  As you can see, the Shadowmap shows the shadow going about halfway out into the street.  


 This is pretty great!  The model aligns pretty closely with the real image! 

 Then I turned to SunCalc and asked the same question, what's the prediction?  Here's what SunCalc shows as the shadow prediction for 9:03AM. 


(I've zoomed in a bit so you can see what's going on.  Note that the predicted shadow doesn't quite hit the street.)  

Well... huh.  That's a little short.  

But by playing around with the time slider, I found that the shadow hits exactly that spot in the middle of the street at 8:42AM, which is 30 minutes off the Shadowmap prediction.  There's that 30 minute difference again!  



Let's recap what we know.  

First, we know we can find online shadow modeling tools that can help figure out what DATE a particular shadow falls on a given place.  By tweaking the dials, we can even figure out exactly what TIME that shadow falls on that place.  

In our case, we wanted to know when the Skytree shadow fell just above the Sumida bridge.  If we use the Shadowmap tool, we can quickly figure out that it's there at 11:12AM on Jan 29, 2020.  By playing around a bit more, we can ALSO find that it fell on that same spot at Nov 12, 2020 at 10:14AM.  (You can do the same thing with Suncalc, just add 30 mins.)  

That's amazing, but why is it true?  

Here's why: 



This is my sketch of the analemma for the Skytree shadow, as computed by SunCalc (and drawn by me).  The analemma, in red, shows the path of the tip of the shadow at 10:42AM each morning over 14 weeks.  (I grabbed the location of each point for the first of each month, then added two more points to get a smoother curve.) 

Then, I plotted the curve of the shadow during the day for Nov 12 and Jan 29.  As you can see, on both those days, the shadow passes over the middle of the Sumida bridge, just at slightly different times.  

If you plot the shadow curves for other days between Nov 12 and Jan 29 (say, Dec 21), those curves will be well to the north of Sumida bridge.  After Jan 29, the curves will be well to the south of the bridge.  

That makes complete sense to me now.  Shadows grow longer during the winter until the winter solstice (Dec 21) and that will put them north of the Sumida bridge.  After Dec 21st, they start growing shorter again, until midsummer (at the bottom of the analemma).  

The bottom line answer?  There are TWO days in the year when that photo could have been taken:  

   Jan 29            11:12 AM 
   Nov 12            10:44 AM 


Whew!  That took a bit of figuring. Thank heavens for online shadow calculators!   

But I STILL don't know why the closed form solution (all those math equations) didn't work out.  Time to break out a pad of paper, a pencil, and do a little trig.  


  
Stay tuned for the exciting conclusion to our 4 part Search Challenge! 
I'll try to push this out on Friday (Oct 22). 
Will write up a NEW SRS Challenge tomorrow (Thursday, Oct 21, 2021).  




As always, I'm Searching On!  

Thursday, October 14, 2021

Answer (2): Time and date from a shadow?

 Can you actually get the time and date from just a shadow? 


This is part 2 of the time&date from a shadow Challenge.  In this post I talk about some of the workarounds I tried in order to get an accurate answer to the question.  This is, in a sense, a record of my exploring various other ways to get to the answer.   

-----------


Remember this image?  


The Challenge I posed was: 

1.  What day and time-of-day was this photograph taken?  

In our last post, I tried doing a bunch of math that I picked up from an Instructables page.  It promised that if I knew the tower location and height, I could run the math and learn the time and date.  But when I tried that, it didn't seem to work out--the math-based prediction didn't agree with 

I used the SunCalc tool to see what they predict the shadow of the Skytree to be on March 21, 2020 at 8:21AM. But both SunCalc and the ShadowCalculator said that the shadow for that day would be about 300 degrees.  (See the simple compass rose below to see why it's 300--just about due Northwest.)  


But 300 degrees is about 65 degrees off from the math model prediction... that's about as wrong as you can get, so something's very wrong. with either the model or my math. 

When this kind of thing happens (and it happens a fair bit when doing online research) and nothing seems to be working, my fallback is to ask myself: 

How ELSE could I find this information?  

A couple of ideas came to mind... 

Look for photos of the Skytree Tower with associated times and shadows so I can get some ground truth about what the shadows look like, when, and where. 

Or... Search for a live webcam in the Skytree Tower area and use THAT as a source of timestamped images.  

So... how can I find time stamped photos?  Here's what I tried.  Foreshadowing: None of these approaches worked! 

A.  Google Streetview:  Here's a view of the Skytree from the Sumida bridge (which is where the tip of the shadow falls).  GREAT IDEA... except you can't get the exact date or time.  You can see the month and year shown at the very bottom of the image, but that's not quite precise enough.  


To make things worse, Tokyo seems to have a lot of cloudy days.  I tried walking around in the Streetview to see if I could find anything good, but I couldn't.  It was either cloudy OR the shadow I wanted to see wasn't visible.  

B.  Aerial images:  Using a query like [Tokyo aerial images] it's easy to find lots of images, but many aren't at a high enough resolution, AND/OR they don't have a good time/date stamp. 

C.  Satellite images:  It's fairly easy to find satellite images, but getting one at a high enough resolution was difficult.  The Japanese Space Agency has lots of great image with decent time stamps (e.g. from the Himiwara satellite), but the resolution is low.  Beautiful images, just not enough detail.   (An aside:  I suspect there are free high-res satellite images WITH time-date stamps, but I haven't found them yet.  If anyone wants a sub-challenge, let me know!) 

I spent a couple of hours searching, but didn't find anything good.  In particular, I checked Google Earth, and there are good pix there, but there's no good way to get the TIME of the image.  

Skytree Feb 7, 2020

Skytree June 7, 2021 
(ignore the apparent tilt--that's an satellite perspective artifact)


Skytree Jan 9, 2018


D.  Time lapse photos:  Easy to find, but mostly of the Skytree as the sun sets, or at night.  While artistic, I wasn't able to find anything that was helpful.  

E.  Amateur photos.  Flickr (and similar amateur aggregator of pics) have great images.  And while I mostly believe the date stamps from the EXIF metadata, I can't believe the time stamp (several of the ones I found were for 1AM, which obviously can't be right.. such as this one below:  

A fragment of a larger image by Graffsma taken Jan 3, 2016. The time stamp on the image metadata is 1AM, which probably is the time at whatever the camera's home location is... 

HOWEVER... I had a realization:  

By using SunCalc, then dialing in the date of the image (Jan 3, 2016 for the Graffsma photo) and then CLICKING and moving the time slider at the top of the page, I can see that shadow looks like it was being cast around 11AM (Tokyo time).  Like this: 



And THAT gives me an idea!  What if I just explored with Suncalc until I found the date/time when the Skytree shadow would appear the same as in our image.... that is, where the tip of the shadow just crosses the Sumida bridge?  

(I actually recommend doing this as a way to get a sense for how it all works--by changing the date and time, you can get a very good feeling for how the shadows move as the date and time changes.)  

Here's what I found for Jan 29, 2020 at 10:42AM, a shadow model that looks just like our original photo: 

Using Suncalc, I can find a time/date that shows the shadow just over the Sumida bridge!  
I just read the time/date off the panel on the left side. 
 


And with even more playing around, you'll quickly realize that the shadow is in the same position on Nov 12, at 10:14AM.  

With Suncalc, it's easy to vary the time, date, and year... by changing the year, I quickly find out that the shadow position varies a tiny amount.  As this scale, I can't quite tell what year it is, but from other evidence, we can tell it's within the past couple of years.  

Bottom Line:  By using an online tool, you can search for a shadow that matches the one in the image!  

Comment: The internet is full of such things--it's a Curiosity Cabinet full of marvelous inventions.  But, as always, be sure to double and triple check.  

In our next episode, I'll try some other tools to see if they cross-check. 

AND... I'll keep working on the math formulas to see if I can't figure out what happened there. 

Stay tuned for the exciting conclusion to our 3 part Search Challenge!


As always, I'm Searching On!  




Friday, October 8, 2021

Answer (1): Time and date from a shadow?

 Can you actually get the time and date from just a shadow? 


This isn't the post I thought I was going to make. As you'll see, this has turned out to be much more complicated than I thought it would be.  So I'm going to do this as a multi-part post--I'll show you my working-out process.  I'll show you what worked, what didn't, and what I learned along the way.  Hope you'll stick with me through the next couple of posts until we get this Challenge solved.  Hope you'll find this as fascinating as I do!  


-----------


In our last Challenge we met the SkyTree in Tokyo, Japan.  (That is, 東京スカイツリー, Tōkyō Sukaitsurī) It's a very tall tower in Tokyo that's known around the world as a landmark.    

And we saw its mighty shadow, extending from the base of the tower to just beyond the bridge over the Sumida river.  


Our Challenge was simple: 

1.  What day and time-of-day was this photograph taken?  

As always, I'm going to start with a simple, fairly open-ended query: 

     [ how to find time and date from shadow ] 

which leads to a bunch of fascinating web pages, each of which tells us how to do exactly this.  (Along with lots of details about the geometry and astronomy involved.)  

A really useful summary of this calculation can be found on the Instructables site at "Time and Date of Satellite Maps."  On this page there's a summary of how and why you can make this calculation.  

When I wrote this Challenge, I knew about this "Time and Date of Satellite Maps" page and the computations involved.  I figured that I'd just do the math, compute the answer, and call it a day.  It turned out to be more complicated than that.  But here's what I did first....   

TL;DR - over the course of the year, the sun moves throughout the sky on a figure 8 path called an analemma.  Here's a NASA analemma image made by taking a photo of the sun at noon each day from exactly the same location: 

P/C NASA. Shot at Corinth, Temple of Apollo

Obviously, this is only the photo from noon, but if there was a tall tower (instead of the Temple of Apollo), you'd see the tip of the tower trace the analemma on the ground.   Like this: 




Or, if you track the sun across the sky, you'll see that every day is just a bit different.  The sun rises in a slightly different place each day, takes a gentle curve across the sky, and sets in a slightly different place.   

P/C Petapixel showing the sun's path across the sky each day. Filmed in Scotland using very clever pinhole cameras (the article is worth reading). 

What this means is that each shadow is unique... except for that one day of the year when the analemma crosses over itself--the sun on that day would cast the same shadow on two different days.  (We'll ignore this for the moment.)  

The Instructables article gives both a satellite worksheet (to collect all the data), and the equations worksheet (to do all the math).  Teachers:  This is a great example of how to do a bit of trig with your class!  

But the key to doing all this math is to first figure out: 

     1.  the lat/long of the tower 

     2.  the height of the tower 

     3.  the length of the shadow 

This is pretty simple to do with a quick search.  

     [ Tokyo Skytree ] 

tells us that: 

     1. lat/long:  35° 42′ 36.36″ N, 139° 48′ 38.52″ E 
                          (or in decimal form: 
35.7101, 139.8107) 

     2. height: 634 m (2,080 ft)

And if we use Google Maps, we can measure the distance from the tower base to just-above the Shimada bridge by using the 'measure distance' feature in Maps: 

To measure the shadow length in Google Maps, Control+click on the base of the tower,
and then click on the point you want to measure (just above the bridge). 
You'll see the distance measure in the popup window at the bottom.  

3. shadow length:  899.7 meters  (2951.9 feet)


We also need to measure the angle between the shadow and true north. In the map, north is straight up, so we can measure the angle by taking the image and drawing a vertical line.  Here, I just drew a straight vertical line (to represent north), and then searched for an [ online protractor ] and pulled the image into that angle measurement tool

You can read the angle of the shadow:  69°


Now we can start to use the equations given in the article.... 

For the angle measurement, we are interested in where the Sun is located, which is exactly opposite the shadow. Measure the angle between due NORTH and the shadow of the landmark.

If the shadow is to the west of North, then the Sun's angular location is:

     Z = 180 - C

If the shadow is to the east of North, then the Sun's angular location is:

     Z = 180 + C

In our case, the shadow is to west of north, so: 

     Z = 180 - 69 = 111

To compute the solar altitude (A) with the length of the shadow (s) and the height (h) of the tower we can compute the altitude (A) of the Sun as:

     A = Arctan(h/s)

In our case, 

     A = Arctan(h/s) = Arctan(0.70467) = 35.17 degrees

(Quick sanity check:  looking at the figure, that seems about right.  I believe that the sun is about 35 degrees above the horizon.)  

How do we get the time of day?  That requires a bit of math.  

First we need to figure out PHI.  That's the angle of the sun along the line it follows across the sky.  That line is called the "solar meridian."  PHI is a measure of how far along that line it is.   It looks like this: 

The black dotted line is the trail of the sun as it moves across the sky from east to west.
Here it's a bit before noon.    Phi is the angle between where it is now and where it rose at dawn.
The black arrow opposite the sun shows the Skytree shadow as seen in the first photo.


The value of PHI is given by this equation: 

     PHI = arctan( (sin(A) * cos(A)) / ( sin(A) * cos(lat) - (sin(lat) * cos(Z) * cos (A)) )

Once you have PHI for the sun's location in the sky, and longitude of the tower, you can compute the time in UTC (Universal Time, Coordinated... which is basically Greenwich Mean Time). Here, both PHI and LONGITUDE are in decimal degrees. 

The value of UTC is the number of hours after midnight.   

     UTC = 12h - (PHI + LONGITUDE)/15.04178

(note: If the landmark longitude is EAST then the longitude is subtracted from PHI.  Luckily, ours is west.  But here's what you'd do if it were east.)

    UTC = 12h - (PHI - LONGITUDE)/15.04178 

Now.... How do we get the date?  

The date is defined by which arc in the sky the Sun moves along between sunrise and sunset.  To figure this out, we need to know the "declination," (that is, how far above the equator the sun has risen) which we'll represent as DEC.  This is calculated from the measurements as: 

    DEC = ArcSin( sin(A)*sin(lat) + cos(lat)*cos(Z)*cos(A) )

With the DEC, you can calculate the Day of the Year the image was taken. There are two possible solutions:

     D1 = 81 + (365/360)*Arcsin(DEC/23.44)

or

     D2 = 81 + (365/360)*[180 - Arcsin(DEC/23.44)]

Note that the way these formula are written, the Arcsin function should be computed to return an angle in DEGREES not in RADIANS.

This gives a numerical day since the start of the year. To convert it into a date you can use an online numbered calendar.  (Search for 

At this point, if you're like me, you're thinking that this is a lot of math to do.  What's the best solution for such a problem?  A:  a spreadsheet.  So I made one to organize all of the math.  Here's a link to my spreadsheet.  And here's what it looks like: 



Note that column D has the values in radians (because the sin, cos, arctan functions all only use radians as inputs).  But I also have all the values in degrees (column B) in case you don't think in radians.  

In any case, once I popped in the values, the spreadsheet did the computation, and out popped March 21, 2020 at 8:22AM.  

Sanity check again:  That's springtime in Tokyo, cherry blossom season, and I'd expect the sun to be somewhat low in the southeastern sky, and casting a long shadow at 8:22AM towards the northwest.  So I expect that at this time/date the shadow from Skytree would look just like the image.  Hurrah!  

But......As always, I thought I'd check my work by finding another source of sun/shadow computation.  I did a search for [ sun shadow calculations ] and discovered several different sun/shadow calculation tools....   

 

I used the SunCalc tool to see what they predict the shadow of the Skytree to be on March 21, 202 at 8:21AM.  Here's what I found:  



Something's really wrong.  That shadow isn't anywhere near the bridge!  

Just to double check this, I used another tool I found, ShadowCalculator:  

Note that it shows time in California time (16:26, which is 8:26AM in Japan).  

So... THEY both agree that the Skytree shadow is in a completely different location than where I predicted with my spreadsheet.  

What went wrong?  
Indeed... what happened here? Since this post is already long, I'm going to write up my next steps in the next post (early next week).  Stay tuned as we hunt for the shadow!  (I'm going into all this depth because this is the kind of thing that happens when you're doing slightly more sophisticated searches--things go wrong.  I want to talk about the process of noticing the errors, and what you need to do to get back on track.)   

Searching on!  




Monday, October 4, 2021

Comment: You've gotta love a tough Challenge!

 I have to admit to loving difficult Challenges... 

... and this one was (I admit) harder than I expected... and much more fun!

A sundial at 7:30AM in Southern California, as seen on one of my runs near Bolsa Chica.


A Cautionary Tale for Teachers and Authors: 

When I wrote last week's Challenge, I actually had found a great resource to answer the "from a shadow figure out the time and date?" I had actually done the work to compute it, and--for once!--I was ahead of the curve.  

But as I was writing up my Answer last week, I did what I always advise people to do:  I looked for a separate source to validate my answer. 

Good news / Bad news:  I found a couple of other ways to figure the answer!  Hurrah!  

BUT... they all gave somewhat different answers!  

I love problems like this. (And I freely admit that this drives most people crazy.  But insight lives in the difficult questions--anyone can solve the easy Challenges!)  

And so... I'm still working away on the Answer to the Challenge SearchResearch Challenge (9/22/21): Time and date from a shadow?

No, Remmij, I don't have the blahs.  I'm actually working on a good answer to a simple problem... and one that I am sure is correct.  

So, if you're a teacher or a writer, here's a maxim:  Check your work... including the cross-checks that you would normally do.  And, when you find multiple different answers, make sure you understand why they differ.  It will make your explanations plausible to your students and readers.  


Still searching!