3.5

Evaluating Sources

Royce Kimmons, Linda Frederiksen, & Sue F. Phelps

Learning Objectives

At the conclusion of this chapter, you will be able to:

Overview of evaluation of sources

Searching for information is often nonlinear and iterative, requiring the evaluation of a range of information sources and the mental flexibility to pursue alternate avenues as new understanding develops. (Association of College & Research Libraries, 2016 [http://edtechbooks.org/-tPv]).

You developed a viable research question, compiled a list of subject headings and keywords and spent a great deal of time searching the literature of your discipline or topic for sources. It’s now time to evaluate all of the information you found. Not only do you want to be sure of the source and the quality of the information, but you also want to determine whether each item is appropriate fit for your own review. This is also the point at which you make sure that you have searched out publications for all areas of your research question and go back into the literature for another search, if necessary.

In general, when we discuss evaluation of sources we are talking about looking at quality, accuracy, relevance, bias, reputation, currency, and credibility factors in a specific work, whether it’s a book, ebook, article, website, or blog posting. Before you include a source in your literature review, you should clearly understand what it is and why you are including it. According to Bennard et al., (2014 [http://edtechbooks.org/-tPv]), “Using inaccurate, irrelevant, or poorly researched sources can affect the quality of your own work.” (para. 4).

When evaluating a work for inclusion in, or exclusion from, your literature review, ask yourself a series of questions about each source.

Evaluating books

For primary and secondary sources you located in your search, use the ASAP mnemonic to evaluate inclusion in your literature review:

Age

Is it outdated? The answer to this question depends on your topic. If you are comparing historical classroom management techniques or building on educational theory, something from 1965 might be appropriate. 

If relying on the book to provide empirical evidence, however, a general rule of thumb would be 5-10 years for education or technology.

Sources

Check reference or bibliography sources as well as those listed in footnotes or endnotes. Skim the list to see what kinds of sources the author used. When were the sources published? If the author is primarily citing works from 10 or 15 years ago, the book may not be what you need.

Author

Does the author have the credentials to write on the topic? Does the author have an academic degree or research grant funding? What else has the author published on the topic?

Publisher

Look for academic presses, including university presses. Books published under popular press imprints (such as Random House or Macmillan, in the U.S.) will not present scholarly research in the same way as Sage, Oxford, Harvard, or the University of Washington Press.

Other questions to ask about the book you may want to include in your literature review:

Evaluating websites

In your research, it is likely you will discover information on the web that you will want to include in your literature review. For example, if your review is related to the current policy issues in public education in the United States, a potentially relevant information source may be a document located on the National Center for Education Statistics (NCES) website titled The Condition of Education 2017 [http://edtechbooks.org/-sgX]. Likewise, for nursing, an article titled Discussing Vaccination with Concerned Patients: An Evidence-Based Resource for Healthcare Providers [http://edtechbooks.org/-zC] is available through the nursingcenter.com website. How do you evaluate these resources, and others like them?

Use the RADAR mnemonic (Mandalios, 2013 [http://edtechbooks.org/-tPv]) to evaluate internet sources:

Relevance

How did you find the website and how is it relevant to your topic?

Authority

Look for the About page to find information about the purpose of the website . You may make a determination of its credibility based on what you find there. Does the page exhibit a particular point of view or bias? For example, a heart association or charter school may be promoting a particular perspective – how might that impact the objectivity of the information located on their site? Is there advertising or is there a product information attached to the content?

Date

Appearance

Reason

What is the web address or URL? This can give you a clue about the purpose of the website, which may be to debate, advocate, advertise or sell, campaign, or present information. Here are some common domains and their origins:

Mike Caulfield (2017 [http://edtechbooks.org/-tPv]), the author of Web Literacy for Student Fact-Checkers [http://edtechbooks.org/-vE], recommends a few simple strategies to evaluate a website (as well as social media):

Evaluating journal articles

It is likely that most of the resources you locate for your review will be from the scholarly literature of your discipline or in your topic area. As we have already seen, peer-reviewed articles are written by and for experts in a field. They generally describe formal research studies or experiments with the purpose of providing insight on a topic. You may have located these articles through Google, Google Scholar, a subscription or open access database, or citation searching. You now may want to know how to evaluate the usefulness for your research. As with the other resources, you are again looking for authority, accuracy, reliability, relevance, currency, and scope. Looking at each article as a separate and unique artifact, consider these elements in your evaluation:

Credibility/Authority

ASK: Who is the author? Is this person considered an expert in their field?

Citation analysis is the study of the impact and assumed quality of an article, an author, or an institution, based on the number of times works and/or authors have been cited by others. Google Scholar is a good way to get at this information.

shows a screen from Google Scholar for a scholarly article. Under the article citation information, the number of times the article has been cited by others is indicated.An example search result in Google Scholar, which lists the Article title (links to article), a brief description, and information about how many people cited the article, related articles, and a web search for the article. The image shows an article titled "The Anatomy of the Grid: Enabling Scalable..." that has been "Cited by 4030"
Google Scholar

Accuracy

Check the facts. ASK:

Reliability/Objectivity

ASK: Is there an obvious bias? That doesn’t mean that you can’t use the information, it just means you need to take the bias into account.

Relevance

ASK: The hard questions:

Currency

ASK:

Scope and Purpose

To determine and evaluate in this category, ASK:

Evaluating Social Media

Although social media (for example, Twitter or Facebook) is generally treated as an object under study rather than a source of information on a topic, the prevalence of social media as communication and sharing platforms must be acknowledged. It’s important to be skeptical of these sources, especially for inclusion in a literature review. However, as with any other web resource,you can evaluate a social media posting for authenticity by asking the following questions:

In Summary

Another way to think about evaluation of sources is to ask the 5W questions:

Locating sources for your literature review by using discovery layers, library catalogs, databases, search engines, and other search platforms may take a great deal of time and effort. Does everything you found and retrieved have value or worth to you as you write your own literature review? If the resource has not met the criteria above and you can’t justify its place in your literature review, it doesn’t deserve to be mentioned in your work. Include high-quality materials that are current, accurate, credible, and most importantly relevant to your research question, hypothesis, or topic.

Practice

Evaluate a Website

Watch on YouTube http://edtechbooks.org/-uE

Using a search engine like Google, do a quick search for a topic that interests you. Select a website from your list of results and evaluate it using the elements of website evaluation listed earlier in this chapter.

Evaluate a Book

Select a subject specific book or ebook that you can access quickly and evaluate it based on the ASAP criteria.

Evaluate an Article

You can practice evaluation using the attached articles. You don’t need to spend a lot of time with the article, but see if you can identify each of the elements of evaluation. Remember the elements of evaluation for articles are:

Example article: Quality standards in e-learning: A matrix of analysis [http://edtechbooks.org/-nXj] (Frydenberg, 2002 [http://edtechbooks.org/-tPv]).

Test Yourself

Your topic is music therapy in kindergarten classrooms in the United States. Which of the two resources would you include in your literature review?

  1. Simpson, Kate, and Deb Keen. “Music Interventions for Children with Autism: Review of the Literature.” Journal of Autism and Developmental Disorders 41, no. 11 (November 2011): 1507-1514.
  2. Bowman, Robert. “Approaches for Counseling Children Through Music.” Elementary School Guidance and Counseling 21, no. 4 (April 1987): 284-91.

Suggested Citation

Kimmons, R., Frederiksen, L., & Phelps, S. F. (2018). Evaluating Sources. In R. Kimmons, Rapid Academic Writing: In Progress: This Book is Currently Being Written. EdTechBooks.org. Retrieved from http://edtechbooks.org/rapidwriting/lit_rev_evaluating_sources

End-of-Chapter Survey

: How would you rate the overall quality of this chapter?
  1. Very Low Quality
  2. Low Quality
  3. Moderate Quality
  4. High Quality
  5. Very High Quality
Comments will be automatically submitted when you navigate away from the page.