Answer: How many students, how many years of school?

The challenge here comes in two parts.  

First, find the data.  Second, create a plot of it in the graphical form of choice.  

To create this chart: 

The number of people in the US with this amount of education, by year.  (In thousands.)  

I did a relatively simple search: 

     [ education data table by year ] 

In my SERP, the 3rd result was the link to the Cenus.gov website.  A quick click took me to the data tables for "Educational Attainment," a phrase that I learned was the correct description of what I was searching.  (I did also check out the website "National Center for Education Statistics," but couldn't figure out how to download decades worth of attainment data.  I gave up on that pretty quickly and turned to Census.gov 

The link labeled "Time series data" looked pretty reasonable to me, and when you click through, you see their graphs and links to their data sets.  

http://www.census.gov/hhes/socdemo/education/data/cps/historical/index.html

You can see there's a graph (on the left) that's pretty much what I asked for (and it would have been okay to stop there).  But I like to get the data for myself, just so I can check what other have done, and possibly re-analyze the data in my own way.  In my case, I summarized all of the post-graduate numbers together.

So I right-clicked on the CSV file and downloaded it to my desktop.  (It's control+click on a Mac.  If you're an Excel user, right-clicking on the XLS file is what you want to do to download it.) 

This copies the CSV to my desktop, and then I can create a Google Spreadsheet and import the data into it.  

I choose to create a spreadsheet rather than a Fusion Table (as Rosemary did) because I didn't think I'd need any of the advanced features of FT.  (It's also easy to import a spreadsheet into a Fusion Table, so it wouldn't be hard to change in any case.)  

I just left all of the defaults about "Import action" and "separator character"to be what they are at the start.  The import process usually gets it right, and in this case, you can see a preview on the right hand side.  

Then, when you click the blue "Import" button, your spreadsheet will look like this.  

My educational data spreadsheet (if you'd like to look at it).  

And at this point, you just select the data (from cell C11 to I69) and click on Insert>Chart 

Note that I've highlighted the "Use row 11 as headers"  AND the "Use column C as labels" buttons.  This is what sets up the chart to plot properly from left to right and create the colors index on the right side of the chart.  

Then all that's left is to modify the labels, and you've got the chart.  


Extra credit: Remember that I'd hoped to find the students with post-graduate years of study as well? 

I was able to find data tables of "degrees granted" by year, but not number in the same format as in the previous data set.  Remember:  those numbers in the chart above are the TOTAL number of people, by year, with that level of educational attainment.

(Yes, I could have looked up the US population for each year and divided... But I took the lazy way out, since I was really curious about the actual numbers of post-graduate degrees attained in any given year.) 

And here's where I went down an interesting rat-hole.  

I did the query: 

     [ number of doctorates awarded US ] 

and found what looked like the perfect answer at the National Science Foundation.  They have a data table "Graduates and post-graduates in science and engineering" (but which includes data on humanities).  They have numbers from the 1940s, so I thought it would be a great resource.  However, when I plot all of the numbers (from both Census and NSF), I see a plot like this:  

It LOOKS like NSF is underreporting (by a factor of 3) OR that the Census is over-estimating the number of degrees awarded each year.  

What's going on? 

I got the NSF data from their data site: https://ncsesdata.nsf.gov/webcaspar/ 

And I got the Census data from their data site.   http://www.census.gov/hhes/socdemo/education/data/cps/2013/tables.html

I've read both of these data tables for a while now, and haven't yet figured out why they're so different.  

The NSF data comes from their survey of graduating PhDs, but they claim a 98% return rate on the surveys.  That missing 2% won't make up the difference in the data.  







Search lesson:  If you want ALL of the data in a time series, search for "time series"--that's the coded language for data collected over time.  

Perhaps most importantly, be prepared to spend time reading over the metadata (that is, the information about the data set) that describes what the data is and how it was collected.  

And when you find discrepancies in the data, embrace the differences and try to figure out what's going on.  (I put both data sets onto a single chart, just to see the differences.)  


Status:  I'll keep working on this over the weekend, but if you figure out why the data is SO different, post a comment.  We'll figure this out together!  


Searching on....