The next step in our framework is to plan your learning assessments. In this chapter, we will discuss the alignment of assessments with course outcomes, different types of assessments, and strengths and weaknesses of assessments. As always, a formative evaluation will be conducted to help you thoroughly think through your design decisions.

Chapter Learning Outcome: I can develop an assessment plan that aligns with my learning outcomes. 

Sub-section outcomes:

• I can plan assessments that align with my course learning outcomes. (Section 4.1)
• I can identify a variety of assessments for use in my course. (Section 4.2)
• I can evaluate the strengths and limitations of my digital assessments. (Section 4.3)
• I can develop an assessment strategy that addresses academic integrity. (Section 4.4)

4.0 Introduction

This chapter discusses the third part of the course design framework: Learning Assessments. We will introduce you to some of the big ideas around digital learning assessment and help you get started designing a digital assessment plan for your blended course. Use the resources listed at the end of each lesson to dive deeper into any of the assessment topics you are interested in.

4.1 Aligning Assessments with Outcomes

In this section, you will begin to address alignment. The best assessments are those that align directly with the learning outcomes that you are trying to achieve. The need for alignment between outcomes and assessments may seem obvious, but it is not uncommon in courses for the assessments and outcomes to get out of alignment—especially over multiple iterations of a course. Video 4-1 below serves as an introduction to this topic.

Learning Outcome: I can plan assessments that are aligned with my course learning outcomes.

Assessment: Blueprint Challenge Part 2 Alignment Table. 

Video 4-1. Alignment and Backward Design (2:53)

What to look for: Why alignment is important and one way to ensure alignment.

As you align assessments with your learning outcomes, consider Bloom's Taxonomy to help guide your choices. Keep in mind that higher-order outcomes require assessments that measure higher-order skills. Figure 4-1 shows some examples of coordinating Bloom's higher-order outcomes and different assessments. However, this does not show all possibilities, so use your own creativity and judgment as you build your assessment plan.

Figure 4-1. Assessment and Bloom's Taxonomy

Created by Alison McMurry, CC BY

You can use an alignment table to articulate the relationship between your desired learning outcomes and the assessments you are choosing. Table 4-1 displays learning outcomes from various disciplines and examples of assessments aligned or misaligned with those outcomes.

Table 4-1. Example of alignment table for possible assessments for various learning outcomes

4.2 Learning Assessments

In this section, you will learn about a variety of learning assessments you may use in your blended course. You will first review the distinctions between formative and summative assessments. Then you will see examples of authentic and renewable assessments, which are particularly relevant in a digital context. Finally, you will have a chance to explore what variety of assessments makes sense for your blended course.  

Learning Outcome: I can identify a variety of assessments for use in my course. 

Assessment: Design Challenge Part 1 Assessment Brainstorming Table—First Column. 

4.2.1 Formative and Summative Assessment

Learning Assessments allow you to measure what a learner has learned from instruction and learning activities. Learning assessments can be formative or summative.

• Summative assessments typically take place after the learning process and have the primary goal of providing an evaluative summary of student learning outcomes.
• On the other hand, formative assessments typically take place during a learning process and have the primary goal of providing actionable feedback to both learners and instructors.  

As you consider potential assessments for your blended course, you may want to consider both formative and summative assessment options as described in Table 4-3.

Table 4-3. Comparing Summative and Formative Assessments

NOTE: Assessments can occasionally overlap, such as if a unit test shows a summative assessment of the unit AND a formative assessment of the course.

4.2.2 Examples of Assessments

Many different digital learning assessments can be used in a blended course.  Table 4-4 lists some examples that you might consider beyond traditional quizzes, exams, essays, and research papers. Using a variety of assessments can enhance the engagement and motivation of your learners.  Additionally, providing a variety of ways to demonstrate learning can cater to and strengthen learners with different learning preferences, abilities, and cultural perspectives.

Table 4-4. Examples of Various Types of Assessments

4.2.3 Incorporating Rubrics

Grading traditional assessments is quite straightforward. Fill-in-the blank, multiple-choice, and matching questions typically found in quizzes and exams can be graded automatically and objectively. To implement any of the alternate assessments listed above, the use of rubrics makes them assessable. Rubrics are tools we use to measure learner performance on an assessment. A rubric identifies and outlines specific criteria that are important. This allows the instructor (or TA, learner, or peer) to give specific, helpful feedback on projects, essays, presentations, and other types of assessment that are not automatically graded. See Video 4-2 below for a brief description of the benefits of using good rubrics. Many LMSs and other digital assessment tools often have ways to easily create rubrics.

Video 4-2. Rubrics (3:04)

What to look for: The benefits of using good rubrics.

The two most common types of rubrics are holistic and analytic. They give different levels of feedback at different points of the learning process. There are sometimes different names and labels used for these rubrics, and some may have hybrid elements, but most rubrics are derivatives of either holistic or analytic. Table 4-5 explains some of the characteristics of these types of rubrics. 

Table 4-5. Characteristics of Holistic and Analytic Rubrics

Let's imagine, for example, that you taught a unit on using rhetorical writing to persuade an audience. The learners were assigned to write a short opinion article on a topic related to the university. For a draft of the opinion article, you could use an analytic rubric, which gives feedback the learners can use to revise their writing. For the finished opinion article, you could use a holistic rubric. Table 4-6 shows an example of an analytic rubric, and Table 4-7 shows an example of a holistic rubric. Notice how they address the same characteristics, but in different ways.

Table 4-6. An example of an analytic rubric for a first draft of an opinion article

First, notice that this rubric does not include anything about grammar, spelling, punctuation, or vocabulary. This is a rubric for a first draft and focuses on higher-order concerns, such as topic, content, strength of argument, and organization. A rubric for a third or fourth draft might focus on lower-order concerns, such as grammar, punctuation, and vocabulary.

Second, notice that this rubric focuses on giving feedback to the learners, so they know what to revise and if they need to talk with the instructor for help. 

Third, consider the strengths of this rubric: This rubric assesses if the learners used rhetoric in their writing. It provides specific, useful feedback so the learners can make improvements to their writing. The instructor receives insight about what the learners understand and what topics need to be reviewed in class.

The instructor can choose to either give a completion score in the gradebook for submitting a draft or give points for each criterion and add the points together for a total score. Both are acceptable ways of recording learner performance on the assessment.

Table 4-7. An example of a holistic rubric for a finished opinion article

First, notice that this rubric gives similar feedback to the analytic rubric, but it is less helpful for making revisions. The product is evaluated as a whole, and the most accurate overall score is selected. A learner may have a great thesis statement, but poor topic sentences and poor use of rhetoric and would thus receive the intermediate score of a B (or whatever value the instructor chooses to give that level).

Second, notice the instructor's rubric offers any learner with a score below A one week to revise the paper and improve the grade if they choose. This rubric is for a writing class and focuses on the process of learning to write, allowing students to learn from mistakes. The rubric gives learners a clearer understanding of how to use rhetoric to persuade. The formative feedback students receive from the rubric allows them to learn the writing process rather than focus solely on the final product.

Third, consider the strengths of this rubric: This rubric assesses how well the learners used rhetoric in their writing. It provides general feedback so each learner can understand their grade. The instructor can evaluate what the learner knows and award an appropriate grade along with guidance for potential improvement.

To create your rubrics

You need to decide what you want to communicate to the learner.

If the assessment is formative:

• Do you want to give general or specific feedback?
• What changes would you expect a learner to make after receiving your feedback? 
• How does your feedback help the learner understand the learning process better?

If the assessment is summative:

• Are you only explaining the reasons for the grade the learner earned, or are you also allowing learners another chance to make changes?
• If they can make changes, what changes would you expect a learner to make after receiving your feedback?
• How does your feedback help the learner understand their product better?

The answers to these questions will help you decide which type of rubric might be best, what criteria to use to evaluate the learner's work, and what to include in the rubric.

Within the categories of formative and summative rubrics, there are many ways to structure the rubrics. For more information about rubrics see the resources section of this chapter.

4.3 Strengths and Limitations of Digital Assessments

In this section, you will learn how to evaluate the strengths and limitations of digital assessments. You will also be introduced to how different types of digital rubrics can help with the assessment strategy for your blended course.

Learning Outcome: I can evaluate the strengths and limitations of my digital assessments.

Assessment: Blueprint Challenge Part 1 Assessment Brainstorming Table - Second Column.

4.3.1 Strengths of Digital Assessments

Technology makes it possible to enhance the assessment of student learning. Figure 4-2 below highlights some of the ways in which learning assessment is evolving in a digital world.

Figure 4-2. The Future of Assessment

Source https://tech.ed.gov/netp/assessment/

When deciding whether to use digital assessments or more traditional offline assessments, it is important to consider the strengths and limitations of your different options. Table 4-8 shares examples of how digital assessments can benefit student learning, access, flexibility, and efficiency.

Table 4-8. Some Benefits of Digital Assessments

4.3.2 Authentic Assessment

Authentic assessment engages learners in real-world applications and problem-solving tasks, which require learners to apply their knowledge in real-world contexts. This deepens learner engagement with the content and fosters greater connections between theory and practice. It also increases motivation because learners perceive greater relevance to their own experience and transferability to their world outside of school. Video 4-3 below presents a brief introduction to authentic assessments.

Below are some questions adapted from Burton (2011, p. 25) that can be used to evaluate the authenticity of a learning assessment. 

1. 1. Is the learner required to mimic a professional in the real world? 
   2. Is the learner required to complete the assessment task using resources similar to that in the workplace? 
   3. Does the assessment task produce a valuable artifact or require the performance of a relevant skill? 
   4. Is higher-order thinking or meta-cognition seamlessly integrated with the assessment task? 
   5. Does the learner collaborate with other stakeholders (for example, professionals/learners) when completing the assessment task? 
   6. Does the assessment task produce novel or diverse responses? 

One assessment may not incorporate every principle listed above, but implementing any of them will increase its authenticity. Table 4-9 contains some examples of authentic assessments in different academic domains. Figure 4-3 shows an example from a psychology class.

Table 4-9. Examples of Authentic Assessments

4.3.3 Renewable Assessment

Renewable Assessments are a type of authentic assessment that has value beyond the scope of the classroom. While most classroom assessments are disposable because they have no lasting value beyond the classroom, renewable assessments produce artifacts that have value beyond the classroom and the assessment itself. They are often associated with internships or projects that serve the local community. 

By creating authentic artifacts with a sense of permanence, renewable assessments enable learners to demonstrate their understanding, critical thinking, and practical skills in a manner that can have lasting value and relevance in their respective fields. Table 4-10 has some examples of renewable and disposable assessments in various fields.

Table 4-10. Examples of Renewable and Disposable Assessments

4.3.4 Limitations of Digital Assessments

Digital assessments have some limitations when compared to traditional offline assessments. Below is a list of some of those limitations.

• Academic integrity — Cheating is generally considered to be easier to monitor in an in-person environment.
• Technical challenges — Internet access, outdated computers and software, and other technical issues can be barriers to a good assessment experience for learners. 
• Human connection — When a skill necessitates human connection, a digital format can be a hindrance (e.g., class discussions, learning to communicate in a language, negotiation tasks, etc.)
• Tactile richness — It may be difficult to evaluate tactile objects (e.g., art projects, physically built engineering projects, etc.)
• Equity issues — While some equity issues might be improved in a digital assessment (e.g., the ability to take an assessment at a convenient place and time), there can be significant limitations if success is dependent on internet speed, software compatibility, and other technical requirements.

4.3.5 Compare Strengths of Digital and Offline Assessments

Blended courses allow for assessments in both online and offline modalities. It is important for you as an instructor to consider the strengths and limitations of the assessments that you design for your course. You might consider creating a table like the example in Table 4-11 as you consider whether to use digital or offline assessments for your blended course.

Table 4-11. Example of Comparing the Strengths of Digital and Offline Modalities for Different Types of Assessments

4.4 Ensuring Academic Integrity

Academic Integrity is the ethical and honest pursuit of learning in an academic setting. Instructors have the responsibility to encourage learners to be honest in their academic work as well as to ensure practices that promote academic integrity. This section will address several course design strategies that can help reduce academic dishonesty. We will discuss the following:

• Creating lower-stake, mastery-based assessments.
• Creating authentic, competency-based assessments.
• Incorporating proctoring for high-stakes exams.
• Incorporating originality (or plagiarism) detectors for writing assignments.

Learning Outcome: I can develop an assessment plan that addresses academic integrity.

Assessment: Blueprint Challenge Part 1 Assessment Brainstorming Table — Third Column.

4.4.1 Creating Low-Stakes, Mastery-based Assessments

Many educational situations create or provide unintended incentives to cheat because the consequences of failure are so high. For example, a final exam worth 25% of the course grade could cause students to feel they must succeed at any cost because performing poorly on the exam would have a dramatic impact on their final course grade. High-stakes assessments usually occur at a single point in time, typically at the end of a learning unit. After taking the assessment, learners move on to the next unit even if they have performed poorly and haven't mastered the material. On the other hand, in a mastery-based approach, learners are given multiple opportunities to demonstrate their knowledge or skills. The mastery assessments are less stressful because they focus on learning. Video 4-4 explains what mastery means and provides examples of it.

Video 4-4. TED Talk — Let's Teach for Mastery (10:49)

What to look for: The analogy of building a house and how it relates to our education systems.

The assessments are often formative and diagnostic in nature. There is an expected mastery threshold that learners are expected to reach before they can move on. Learners may take a quiz or an exam and get feedback on gaps in their understanding. They can do additional activities to address those gaps and retake the assessment.  Figure 4-4 depicts the difference between a traditional approach (time-based progression) and a mastery-based approach. 

Figure 4-4. Time-based vs. Mastery-based Progression 

Created by Doug Archibald CC SA

Digital assessments can play an important role in making a mastery-based approach possible.  For example, Section 4.3 identified the following strengths that can make mastery-based assessments more practical.

Real-time Feedback/Feedback Automation — in a mastery approach, getting quick diagnostic feedback on assessments allows learners to seek remediation immediately while the problems are fresh in their minds. 

Multiple Attempts — a mastery approach depends on learners being able to take assessments multiple times. This typically means having a bank of items that measure the same knowledge or skill. Digital tools enable each attempt to draw different questions from the question bank, ensuring that when learners are making another attempt, they aren't just learning the answers to the previous assessment items.

Adaptive Pathways — in some domains, digital software is utilized to give learners adaptive feedback based on their performance on mastery-based assessments. For example, a learner who has missed questions related to Concept A will be directed to different learning activities from a learner who has missed questions related to Concept B.

4.4.2 Creating Authentic, Competency-Based Assessments

Competency-Based Assessments are becoming more common in education. These assessments are like the mastery-based approaches described previously; however, a core idea behind many competency-based approaches is that learners can demonstrate their competency through authentic experiences that they have had outside of the classroom and sometimes before enrolling in the course.  Below are a few examples:

• Computer Science — a learner can use a software program developed previously for a client to demonstrate specific coding competencies.
• Nursing — a learner who was a medic in the military can demonstrate already learned skills based on their military experience and training.

Watch Video 4-5 to understand how competency-based education compares to traditional education.

Video 4-5. Competency-Based vs. Traditional Education (1:54)

What to look for: Three ways competency-based assessment is more equitable than traditional assessments.

Relying more heavily on authentic, competency-based assessments as part of your course grading is another way to strengthen learning and simultaneously deter cheating. While it is not impossible for learners to cheat in authentic assessments, the design and nature of these assessments make it significantly more difficult to do so. Below are some ways that authentic assessments promote academic honesty.

Contextual relevance: Authentic assessments are designed to reflect real-world scenarios and challenges. The questions and tasks presented in these assessments require learners to apply their knowledge and skills in practical ways often in a unique real-world context. Cheating through memorization or using external resources becomes less effective because the questions often demand higher-order thinking and require the ability to analyze and synthesize information in specific contexts.

Complex problem-solving: Authentic assessments often involve complex problems that cannot be easily solved by simply looking up information. These assessments focus on assessing a learner's ability to think critically, make reasoned judgments, and develop creative solutions. Cheating becomes more difficult because the tasks typically require a deep understanding of the subject matter and the application of acquired knowledge to novel situations.

Performance-based tasks: Authentic assessments often involve performance-based tasks such as presentations, demonstrations, or projects. These assessments require learners to showcase their skills and abilities in a tangible and often personal way, making cheating more difficult. 

Assessment process: Authentic assessments often involve a process rather than a single final product. For instance, a research project may require learners to develop a research question, gather and analyze data, and present their findings. The assessment process itself provides opportunities for instructors to observe and evaluate a learner's progress, making it harder for cheating to go unnoticed.

Assessment variety: Authentic assessments can take various forms, including group work, field experiences, internships, and simulations. These diverse assessment methods make it more challenging to cheat, as the evaluation is not limited to traditional test formats. The authentic nature of the assessments reduces the predictability of cheating methods and requires learners to demonstrate their skills in different contexts. Figure 4-5 shows examples of competency based assessments.

 Figure 4-5 Example Competency-Based Assessment - TV commercials

This is an example of learners creating a series of commercials, promoting a local business for a viewer contest, as part of a local TV station's self-promotion initiative.

• Learners were enrolled in the Radio, Television, and Broadcast News diploma at the southern Alberta Institute of Technology (SAIT).

• Learners were tasked to find a local business or not-for-profit.

• Working with this business they needed to write a script that their client approved, and then direct the shoot, editing, and production of two commercials (30 second and 15 second versions).

• The client could freely use the commercial where they wanted, and the learner could keep it for their demo reel (like resume for the video industry).

• The examples were made in 2016. These commercials aired on CHAT TV in Medicine Hat Alberta for about 2 months.

Primetime 30-second spot

Primetime 15-second spot

4.4.3 Incorporating Exam Proctoring

Exam proctoring is the process of monitoring learners as they take an exam. Exam proctoring can happen in-person or remotely using digital technologies. Typically, the monitoring is done for high-stakes, summative assessments to discourage academic dishonesty and ensure that the exam conditions setup by the instructor are enforced. 

In-person Proctoring. In the in-person classroom, proctoring is often done by the instructor who walks around the classroom observing learners to ensure that cheating does not happen and that learner questions during the exam are addressed (see Figure 4-6). Some institutions have a separate testing center where learners can take exams overseen by human proctors outside of class time. 

Digital Proctoring. Digital Proctoring involves using technology to monitor learners and ensure desired test conditions are met while they take an exam (see Figure 4-5). Some common features of digital proctoring software include:

• video monitoring — use of video to record or observe the learner taking the exam in real-time.
• screen recording — recording of activity taking place on the computer monitor during the exam.
• browser lockdown — locking down browser screens or features to prevent accessing unauthorized information.
• identity verification — features such as facial recognition or ID verification that ensure the test taker is a registered learner.
• environment monitoring — detecting suspicious behaviors in the exam environment such as eye movement, head gestures, audio activity, or anomalies in screen activity to identify potential irregularities.

4.4.4 Incorporating Plagiarism Detectors

Plagiarism is when a person presents someone else's ideas or work as their own without adequate acknowledgement. Some examples of plagiarism include:

• copying blocks of text from a website for use in an essay without proper attribution.
• copying code written by another person and representing it as your own work.
• using artwork or images from another author without proper attribution.

Plagiarism detectors are software programs that help identify instances of similarity in written content. The software generates a report identifying places in a submitted text that are similar to other work in the database. Figure 4-7 shows an example of plagiarism software.

Figure 4-7. Photo of Plagiarism Detector Report

Instructors and learners can then use the report to determine if plagiarism may have happened. In this example, 10% of this paper is similar to other sources. Clicking on the match in the report links to a side-by-side comparison of the texts. 

Plagiarism detectors can be used to encourage academic honesty through plagiarism prevention and detection strategies. 

Prevention

Sometimes learners commit intentional plagiarism. They purposefully use work from other people and claim it as their own. Informing learners that their writing will be processed by a plagiarism checking program may be enough to deter them. It is a best practice to inform learners in advance that the assignments will be checked, show them the program that will be used, and explain the generated report. Once they are informed of the instructor's expectations, they may be more likely to act with academic integrity.

Detection

However, it is more common for learners to commit unintentional plagiarism because of carelessness or ignorance. Learners usually think they have made sufficient changes to the borrowed information, or they think they have given sufficient acknowledgement to the original author, when they have done neither of those things. 

Allowing learners to upload drafts to a plagiarism detector and teaching them how to use the report to revise their writing can be a very beneficial learning experience. The learners can see where they have insufficient or inappropriate quoting, paraphrasing, or summarizing, and learn how to do it better in the future. The detectors do not scan for citations or references, so the instructor will need to teach the learners how to review their own writing to verify adequate in-text citations and a correct reference list.

Use of Generative AI

Using generative AI, such as ChatGPT, can be a helpful tool for brainstorming and drafting. Many institutions are still trying to develop policies around its use to help learners to know when it is appropriate and inappropriate to use it. In some situations, the use of generative AI can be a form of academic dishonesty because it doesn't necessarily reflect what a learner knows about a subject. The plagiarism detector will not flag text generated by AI as plagiarism because the text will be unique. Be aware of this, come up with a plan that addresses academic integrity, and communicate your expectations to your learners.

Conclusion

Chapter 4 has helped you think through aligning your assessments with your learning outcomes and introduced you to types of assessment, strengths and limitations of the different types, and ways to address academic integrity. You have completed the foundational work and can now create your digital assessment plan.