How to find anything #5: (part 3/3) Assessing Credibility of News Sources

 #8.3  Assessing Credibility of News Sources 

Foreword:  This is part two of our chapter on finding news (and late-breaking information).  Mario Callegaro and I have been writing this to let you know some of the best practices in how to find online news.  Here we talk about that important step--how to assess the credibility of a news source.  How do you know when you should believe a new source?  

  

When you’re looking for news sources you’ll naturally want to find credible sources.  The hard here is what’s credible… and how can you spot a credible source?  In other words, do you trust a source to be accurate even when you can’t verify the claim yourself?  That’s why you want to use credible sources: that’s the trust you place in the source.  

In other words, credibility is a measure of how much you trust the source to be accurate and honest about their reporting.  Every news provider will make mistakes, but a credible provider will acknowledge them openly.  

Brands are all about trust. That trust is built in drops and lost in buckets. 

             – Kevin Plank, Feb 21, 2014 – USA Today interview.  

So, how can you identify a credible source?  

I hesitate to give any checklist of features to look for two reasons:

1. Those lists tend to become out-of-date rather quickly, especially when different players start to game the system to appear more credible than they really are; and 

2. The credibility of a source is built up over time and can’t be determined by examining a single article.  

(But if you want a checklist, here are a couple: the CRAP checklist, MLA checklist CXL’s checklist, Mike Caulfield’s SIFT list.  You know how to find more.)  

How then can we identify credibility in the news we seek?  

Here’s the truth about credibility:  Identifying a credible site takes time.  Credibility, like trust, is easy to lose and difficult to gain.  Yet that’s what you’re doing as you read the news—you’re learning which sites have credible content.  What all those checklists are saying is that credibility has many hallmarks, but none of them are completely diagnostic—they’re brush strokes that are trying to help you assess how believable you find the source.  That is:  

Credibility is created by sustained accurate and honest reporting by a news source. When a source is credible, the expectation is that the information being presented is accurate, and implicitly, you don’t need to fact-check every detail.  

A  particular article might tick all the boxes in a “credibility checklist,” it might have a byline, have citations, and present the information in a reasoned and verifiable way.  It might have quotes that you can trace back to the original source, and present the information in a sober non-combative way.  Those are all great features to have.  But credibility is more complex than the presence or absence of those individual characteristics.  

Recognize that news sources often have a composite nature (e.g. large paper will run editorials from different perspectives, and individual writers have different reputations).  News sources are often not monolithic and through-written by an individual.  Instead, the op-ed might well be written by an author with a very different perspective than the editorial stance of the site.  Bear in mind that a single article might well be non-credible, but be found in a credible journal.  (And vice-versa.)  

So, what can one do to assess the credibility of a source?  Here are some aspects to consider:  

Reputation is variable: Credible sources tend to be well-known, they tend to cite each other, and they have a good mutual reputation.  But bear in mind that different sources can have very different reputations in different communities.  You might find the Wall Street Journal to be a credible source of economic information, but not to be a trustworthy advocate for economic policy positions.  (I trust my Mother’s advice about our family’s recipes, but not so much her thoughts on the economy.)  

Check different versions of the story:  When looking for a source of information on a given story… search for a range of sources in the area of interest. Get different perspectives on the story, and then see how your source stands up in the reporting.  Are the facts consistent across different stories?  How do different news sites tell the same story?  The differences between stories by news source is often a useful guide to how to understand the site’s credibility.  Often, a news aggregator ( news aggregator, feed reader, news reader) will put different versions of a story side-by-side allowing for comparison reading.

A deep insight about news:  Remember that “facts” of the moment are often transitory and are constantly being updated as new information is discovered.  Keep this in mind as you wade through the news--our understanding of facts, what's going on, and how to interpret what we hear is constantly under revision.  

Credible sites admit to errors:  A site (or author) that never makes a mistake isn’t doing reporting, they’re writing religious tracts.  Check for updates, errata, corrections on the site.  Credible news outlets admit, identify, and correct their mistakes.  Non-credible ones don’t.    

Pay for subscriptions: As you identify credible sources, we encourage you to pay for subscriptions to quality news sources (you will have your own list, but for example, my paid subscription list includes the New York Times, Wall Street Journal, Washington Post, The Atlantic, The Economist, The New Yorker).  We also have some international sources such as BBC and Al Jazeera. 

Fact-checking sites:  When determining credibility of news sources, it’s often useful to have some some fact-checking sites in your credibility toolbox (e.g. Factcheck.org, Politifact.org, Washington Post Fact-check, AP Fact Check, FullFact.org, or Snopes.com).  Remember that Wikipedia maintains a fact checking page by country.

Credible specialty sites:  If you’re interested in speciality topics, you’ll probably want to find reliable sites that cover only that topic area.  You might be interested in the politics or your city/state/region/country, so we encourage you to spend some time looking at different perspectives on that topic.  This advice is as true for math as it is for political stories.  Different points of view—even about topics that don’t seem to have different perspectives (e.g. math)--can be useful if only as a way to get different ways of telling the same story. For a completely different point of view (for us), we will sometimes check the Australian news services News.AU and the ABC (AU)

Finally, note that credibility is potentially transient—what you find a credible source NOW might not be a credible source next year. (They could have changed, or YOU could have changed. In either case, keep checking!)  

 

The Can-you-tell-the-story? test

One of the simplest ways to self-assess whether a story is believable or not (and whether or not you understand the story and its sources), we often just try to tell-the-story to someone else.  Imagine that you’re going to tell this story to your mother (or a friend, or whoever you trust).  Can you actually tell the simplest possible version of the story and feel as though you can give an honest account of the story?  

It’s often the case that if you actually try to re-tell the story (or write it out), gaps in your understanding will often appear.  And—in particular—you might well come to understand whether or not you should trust your news sources.  Does the story make sense, or are there hard-to-follow claims?  Do you understand all of the terms you’ve used in telling the story to your trusted friend?  Do you trust the sources that the news provider used in telling their story?  

All of this will come to the front as you retell the tale.  Can-you-tell-the-story?  It’s a simple sanity check.  If you can’t re-tell the story in a way that YOU believe, then you have to question the credibility of your sources.  

What we do to keep track

Mostly, we collect reputable sites and authors.  As we recommended in the previous post, keep track of the news sources that you find especially good.  Some useful sites that we follow and have in our list of good sites are Techmeme (technology), Metacritic (reviews of movies), (e)Science News (for a feed of science news) and ScienceNews (for another perspective on science news). For health news we follow Healthline, or for business news Bloomberg, for example.  Ask around within your community to learn what sources your friends and colleagues consider reliable and useful news sources.  

Of course, your mileage may vary: it’s ultimately up to you to find your own useful and believable sites (and then keep track of them).  

You can find your own news sites by searching for [ news site <topic> ] For instance, [ news site classical music ] will lead you to a bunch of newsy sites on that topic. 

We also learn about authors as we read.  When you find good (and credible) articles, take note of the author.  Good writers tend to write multiple articles within a domain of expertise.  Note their names and seek out their reporting.  

 

___________________

And, just for extra, bonus points and a very different way to find news that you care about, check out the… 

 

Wikipedia Current Events Portal:  The Wikipedia current events portal is an interesting way to get an overview of the world’s stories without much commentary.  There's one line for each major news story, including many international stories.  There's even an archive feature (see red circle in image below) that can take you back to Jan 1994, although depth of coverage drops over time.  

 

Search on!