5.3 Evaluating Sources

The quality of your overall message is greatly affected by the quality of the evidence you use. If you use excellent evidence, you are well on your way to composing an excellent message and meeting your communication goals. But if you use weak evidence, your entire message may be negatively affected. That’s because if your receiver identifies some of your evidence as weak—even if it’s only one small piece—it may cause your entire argument to be called into question. Therefore, it is of utmost importance to use high-quality evidence.

The CRAAP test shows you a series of questions you can ask yourself to determine whether you a piece of information is trustworthy and useful.

The CRAAP Test

Sarah Blakeslee and the librarians at California State University, Chico, came up with the CRAAP Test to help researchers easily determine whether a source is trustworthy. You can download a handout that explains the CRAAP test here. One more “R” has been added for you to consider.

There are six parts to the CRRAAP Test:

• CURRENCY: When was the information published? For some topics, it’s okay if you use an older source. For example, if you want to know what DNA is, it’s okay if the source was published five years ago. But, if you’re researching what the latest DNA discoveries, a five-year-old source wouldn’t be helpful. There is no arbitrary rule on how new something must be to be considered sufficiently recent. Instead, the principle here is that whatever evidence you use should be recent enough that newer information hasn’t supplanted the information you’re using. For instance, if you want to update your receiver about your social media marketing campaign, data that are more than a week old may be too old to be useful. You’ll need to get more recent data. But if you are estimating costs for purchasing automobiles or advertising space, prices from the last year are probably sufficient. And if you are using evidence from a research study on consumer psychology and how mood affects buying behaviors during the holidays, a study that is 10 or more years old might be perfectly recent. As you look for evidence, track down the most recent sources available.
• RELEVANCE: Does this information meet your needs? For example, an article aimed at educating young children about DNA would probably not be a relevant source if you work for a tech firm and are writing a report about whether to acquire some DNA technology. It’s okay if a source isn’t perfectly relevant to your research question, since that’s where analysis comes in. For example, if you were researching the Housing Crisis in Vancouver, you might read about how expensive cities like San Francisco and Hong Kong are dealing with their own housing crises in order to get ideas.
• REPRESENTATIVE: Another important quality marker for evidence is its representativeness, or how accurately it reflects or represents something else. There are two important facets of representativeness. First, evidence should accurately represent the broader domain being described. Averages and midpoints tend to be more representative than highs and lows. Views held by a majority of people are more representative than views held by a single person. For example, if customer reviews of your new product line are half positive and half negative, but you only include the positive comments when reporting to your leadership team, that would not be representative. Instead, as a competent business communicator, you would use evidence that represents the broader reality of mixed reviews and, of course, adjust your claims accordingly. Second, evidence should accurately represent the intent and content of the source. Statistics and quotations are particularly prone to being misrepresented, whether that is by presenting only a portion of what was found or said, mischaracterizing the broader intent, or not providing enough context to make sense of the evidence. Consider the difference here:

  Partial Quotation: When asked about recent accusations of sexual harassment in the organization, CEO Paul Seaton said, “It’s not my problem.”

  Full Quotation: When asked about recent accusations of sexual harassment in the organization, CEO Paul Seaton said, “It’s not my problem. It’s our problem. We have to work as a company to fix our culture.”

• AUTHORITY: Who wrote this information? Your source should be written by someone who has the authority to speak on the matter. For example, you might come across a blog that offers natural health remedies for cancer or depression written by someone who says they’re a doctor. If they’re a doctor because they have a PhD in English Literature, however, then they wouldn’t have the authority to offer medical advice. (You would, however, be able to trust their opinion on depictions of natural remedies in novels). To determine how trustworthy an online source is, you can also look at the URL. If it comes from a .gov or .edu website, you would probably trust it.
• ACCURACY: How reliable or trustworthy is the information? Specifically, you should examine how the source uses evidence. Does the source link to other trustworthy sources? Does the source support its claims with evidence? How reliable is that evidence?The most essential quality criterion of evidence is that it must be correct. Evidence marred by miscalculations, misrepresentation, or other inaccuracies can misguide decision-making, damage your reputation, and negatively affect business. So take time to “fact check” your evidence. Here are two key questions to guide you.
  First, “Does it make intuitive sense?” For instance, assume you do a salary analysis for your full-time employees and see that your lowest-paid employee makes $8,000 per year. An intuition check should tell you there is a mistake somewhere, as a full-time, minimum-wage employee would make much more. If you find evidence that doesn’t pass the intuition check, you should either correct the mistake (if that is within your control) or discard the flawed evidence and search for another source. Second, “Can it be confirmed?” If evidence is accurate, it should be able to be confirmed. Confirmation can occur in multiple ways. You could rerun analyses and double-check your own calculations. You could verify facts with a knowledgeable colleague. You could seek out other external sources to corroborate the evidence.
• PURPOSE: Why was this information published? You should be able to identify how the author benefitted from publishing this information. Specifically, consider how the author makes money from putting this information out there. Sources that make money aren’t necessarily untrustworthy, but following the money gives you a clue if the author in question has be incentivized to support a position. In many ways, all evidence is somewhat biased. Even the act of your choosing what evidence to include and what evidence to exclude introduces bias. Even though you can never be completely bias-free, you should strive to be as unbiased as possible. In business, an important type of bias is conflict of interest. A conflict of interest occurs when someone’s impartiality may be influenced by a professional or personal interest. For instance, salespeople may not present a fully unbiased account of their company’s services if they will financially benefit from making a sale. Or managers may not be able to write impartial performance reviews for employees who are also friends. Simply asking, “Who wrote this?” or “Who provided this information?” can help you determine if the evidence is reasonably free from overt bias. For instance, a source such as Wired magazine is going to present less biased product reviews than websites for the companies that make the products.