Usability

In addition to the qualities of validity and reliability, when creating or selecting a test to measure specific learning objectives, you must also consider the usability of the assessment results and the practical aspects of testing. These aspects should not be an afterthought but should be carefully considered when designing or selecting an assessment.


Usability, within the context of assessment, embodies a crucial yet often overlooked facet of assessment. Usability refers to the practical considerations of assessment. The practical aspects associated with a test's usability typically center around issues of administration, interpretation of results, cost, and availability. But should also consider fairness, transparency, and authenticity. The usefulness of the results we get from a valid test are significantly undermined if they cannot be easily obtained, are difficult to interpret, are not relevant for some groups of students, or cannot be used for some other reason. Aspects of usability that need to be considered include:

Ease of Administration:

One aspect of testing that might make an assessment less useful is whether it can be easily administered. A test should not be excessively difficult for a teacher to manage or for students to take. For example, some specialized assessments require training and the certification of test administrators. The required training increases the administration cost, and although these tests may yield valid results, the required certification may make the test less likely to be used. Other tests need to be administered individually. This adds to the administration costs and the time needed to carry out the assessments. In addition, some assessments have specific resource requirements that must be met before the exam can be administered. These resource requirements may result in an accessibility problem for some students.

For the students, administration issues may involve not having clear directions or an appropriate amount of time to complete the exam. If students taking the exam find it hard to understand how to complete the questions, this could be a problem. Students also need enough time to take the exam, but the test should not be excessively long. Likewise, over-testing and making students take too many exams might result in exam fatigue and constitute a usability problem.

Transparency: A useful assessment must be valid, but it must also serve a clear purpose. Rarely can an assessment be valid for more than one purpose. Transparency in an assessment context involves clearly communicating to all stakeholders, including students, the purpose of an assessment and how the results will be used. Transparency requires that a test aligns with specific learning objectives. A test's perceived utility and relevance are enhanced when students understand why they are being tested, what is expected of them, and the criteria for evaluating them. This clarity can aid in student preparation and foster a sense of fairness and accountability in the assessment process. Providing a clear overview of the content and skills covered, the test design and question types, the scoring process, and how results are to be used allows students the opportunity to perform to the best of their abilities.

Ease of Interpretation and Application: Results from assessments that are difficult to interpret diminish their utility and increase the risk of misuse. Assessment results are intended to provide insights that enable teachers to tailor instruction, guide student learning, and assess progress. In addition, the results need to be presented in a way that makes sense to stakeholders. When educators are presented with assessment results that are excessively complex or unclear, it hampers their ability to make informed decisions. If you need a psychometrician to explain the result of an assessment to teachers, students, and parents, you have a usability problem.

For example, consider a standardized reading comprehension test given to a class of fourth-grade students. If the test results are reported only as scale scores without further explanation or context, stakeholders might struggle to understand what the scores mean and how to use them. Even results that most teachers think they understand, like grade equivalent scores, may need to be explained to students and parents. A lack of clarity can prevent teachers from tailoring their instruction effectively. On the other hand, if the test results are broken down into detailed components, such as vocabulary understanding, ability to infer meaning from context, and recognition of the main idea, and are accompanied by suggestions for targeted instructional strategies, teachers can use this information to address specific areas where students need improvement. This approach makes the assessment results more interpretable and directly supports individualized instruction.

 The presentation of results is crucial for ensuring they are actionable. Results should be communicated in a manner that is understandable to educators, students, parents, and school administrators. Properly presenting test results might involve using visual aids like graphs and charts to depict student performance, providing comparative data to show progress over time, and including practical recommendations. By presenting data in a clear and actionable format, assessments become a powerful tool for driving educational improvement and fostering a collaborative environment focused on student success.

Fairness: Fairness is intrinsically linked to the validity and utility of an assessment. An assessment that is not fair cannot be considered valid, as fairness encompasses the equitable treatment of all examinees in the assessment process. This means that assessments should be free from bias, accessible to all students regardless of their backgrounds, and should accurately reflect the diverse ways in which students may demonstrate their knowledge and skills. Fairness also involves accommodations for students with special needs, ensuring that the assessment measures what it is supposed to measure rather than the students' disabilities or linguistic backgrounds. An unfair assessment compromises its validity and utility, as it fails to provide a reliable measure of all students equally.

If certain groups of students are systematically disadvantaged by the design or administration of the test, its results will not accurately reflect that group's actual abilities. Fair assessments give all students an equal opportunity to demonstrate their knowledge and skills. Assessments that are biased against certain groups or that fail to provide accommodations for disabilities lack utility for those students and diminish the quality of the assessment.

Authenticity: Authenticity in assessment refers to the extent to which an assessment reflects the application of knowledge and skills in real-world situations. The disparity between theory and practice in assessment can significantly affect its utility. Authenticity is an especially relevant factor for performance assessments. Knowing about something cannot be considered as evidence that an individual can skillfully accomplish a task. An assessment grounded in theoretical knowledge but disconnected from the practical application of that knowledge or skill may fail to provide learners with meaningful insights into how their learning applies to real-world contexts. Conversely, assessments that incorporate authentic tasks or simulate real-life situations enhance the relevance of the learning experience and increase the likelihood that the classroom learning transfers into real-life situations, thereby increasing the utility of the assessment. Authentic assessments bridge the gap between classroom learning and practical application, making them more useful for students and educators.

Chapter Summary

  • When creating or selecting assessments to measure specific learning objectives, in addition to validity and reliability, it is essential to consider the usability of an assessment.
  • Usability focuses on the practical aspects of testing.
  • Administration: A usable assessment should be reasonably easily administered. Factors affecting administration include cost, need for specialized training, time and resource requirements, and student accessibility issues.
  • Transparency: Clear communication about the purpose of an assessment, its alignment with learning objectives, and how results will be used is essential.
  • Transparency enhances the perceived utility and relevance of an assessment.
  • Ease of Interpretation and Application: Assessment results must be easily interpretable to be useful. Complex or unclear results can hinder educators' ability to make informed decisions about students' abilities and how to help them learn.
  • Presentation of Results: Results should be presented in an understandable and actionable manner for all stakeholders, including educators, students, parents, and administrators.
  • Fairness: Assessments must be equitable, free from bias, accessible, and accurately reflect the knowledge and skills of all groups of students. Unfair assessments compromise reliability and utility.
  • Authenticity: The extent to which an assessment reflects real-world applications of knowledge and skills can affect its utility. Assessments that incorporate authentic tasks or simulate real-life situations can be more valuable and relevant to students because they bridge the gap between classroom learning and practical application.

Discussion Questions

  1. Describe a specific test that might be valid but has usability issues. Describe how the practical issues of the assessment limit its usefulness. 
  2. Describe a situation where fairness is an issue for a specific test. Discuss ways that the usability of the test may be improved.  
  3. Describe a specific assessment and how the results are shared. Provide an example of how you might present these assessment results in a way that is useful to students and parents.   

This content is provided to you freely by EdTech Books.

Access it online or download it at https://edtechbooks.org/Assessment_Basics/utility.