Chapter 6 | Assessment

Student Participation and Progress

Introduction

Assessment is an important part of both face-to-face and distance education. Adult educators use assessment for several reasons: to determine an appropriate placement for a student before instruction begins, to gauge learner progress in the course of an instructional sequence, and to measure how well a program of instruction is working. Determining placement and measures of program effectiveness are often accomplished using standardized tests (e.g., TABE, CASAS, and BEST Plus) or assessments developed by a program. Gauging learner progress can be accomplished by using a combination of formative and summative assessment strategies.

Why Assess?

Provides diagnostic feedback

Helps educators set standards

Evaluates progress

Relates to a student’s progress

Supports student self-evaluation

Supports teacher self-evaluation



Formative Assessment to Gauge Student Progress and Guide Instruction

Assessing student work on a regular basis provides both the teacher and the student with a sense of the student’s progress, indicates strengths and areas for improvement, and helps the teacher plan appropriately to meet the student’s needs. This formative assessment is part of the process of a learning sequence (Bakerson, Trottier, & Mansfield, 2016; Popham, 2011). Formative assessment can be structured using rubrics, quizzes, or observation protocols. It might also be less formal, quick comprehension check questions asked throughout an instructional period or exit tickets turned in at the end of class (Sparks, 2020). Assessments are valuable for students because they provide a way for them to gauge their progress toward meeting goals

Tips for Doing Formative Assessment in Distance Education

Collect data over time. Formative assessment is a process, so you should collect evidence of learning throughout the semester.

Require students to submit evidence of learning.

For example, you might have students submit reflection videos using Flip, formerly called FlipGrid, or send photos or screenshots of their progress. Learners can use Google Jamboard or Google Sites to develop a portfolio of their learning progress. Ask students to complete regular self-assessments by having them indicate progress by completing a weekly survey that lists expected progress markers; give them opportunities for reflection on that progress (Miller, 2020).

Provide feedback.

Provide written feedback on shared documents or discussion boards. If you have some face-to-face time, provide oral feedback through videos or sound recordings. You might use breakout rooms for students to give feedback to each other. In a distance format, you can use a discussion post or collaborative work in a Google Doc for students to provide feedback during established time frames (Miller, 2020).

Include comprehension checks in remote face-to-face class meetings.

Embed quick comprehension checks in your instruction (Miller, 2020). Use Yes/No buttons in your webinar tool, short question response prompts in chat, or Handswers (an engagement strategy where students are prompted to hold up a number of fingers to select a response). Get creative and embed questions directly in your presentation slides. For example, using a slide like this, you can have students add responses to quick feedback questions.

Image of a Zoom poll asking learners to rate their comfort levels

You can also create class slides using Pear Deck integration in Google Slides. This extension for Google Slides makes it possible to embed questions for your students to answer as you give a lesson. 

Connect personally.

Limiting teaching and learning to remote or distance contexts can feel isolating. A recent study of adult education instructors showed that most instructors relied on reaching out to learners personally between video classes—often via a phone call (Belzer et al., 2020). You can make the most of these conversations by following these tips:

  1. Prepare for the call; know what you want to ask about. Plan questions that will inform you about where students are in their assigned work and what problems they might be having. Decide ahead of time how formal you want the call to feel. If you have particular learning objectives that you need to assess, plan out the questions ahead of time.
  2. Keep track of what you learn in these calls. Use a tracking sheet such as this example that helps you maintain records of learner progress around their goals.
  3. Include questions about how students are experiencing the distance education format and activities. Ask about what’s working or what activities are particularly challenging. Ask for suggestions on what changes the student sees as useful.

Use what you learn.

Adjust your instruction based on what you are hearing from your students. Gathering data, organizing it, and reviewing it will show patterns about where your technology and activity choices are not working or where you might need to add supplemental resources for more content.

Summative Assessment to Measure Learning Over Time

Interim and summative assessments both measure learning over time. Interim assessments show individual student progress toward a set of standards. These might be considered summative tests of a chunk of content. They happen periodically, like in the middle of a curriculum unit. They are also somewhat formative because teachers can adjust instruction for the rest of the unit or block of time (Sparks, 2020).

Summative assessments compare a student or group of students against a set of standards. Though they do show individual student progress, they also measure the efficacy of instruction. This assessment occurs at the end of a unit or course or program year. Summative assessments are standardized in order to support comparisons among students or groups of students (Sparks, 2020).

Tips for doing summative assessment in distance education

Do not assess everything. Your list of standards is likely longer than what is possible for you to assess in the time you have with students. Follow this R.E.A.L. guide to determine what to prioritize (Many & Schmidt, 2014):

Make use of performance assessment.

Performance assessments require application of knowledge and skills, rather than just rote recall or demonstration of them. They often result in an end-product like a presentation that is informed by more than one subject and crafted by drawing on a range of technology skills. There is generally no single correct answer, but evaluation is done by using a rubric (Miller, 2020).

Take into account differing access to technology.

Don’t assume that students will have the same access to technology. Because access might be limited to specific times, have students take the assessments during a remote face-to-face class session. Also allow for oral assessments that might be delivered over the phone. You could also have students complete handwritten activities that they photograph and text to you (Miller, 2020).

Examples of Assessments Possible in Distance Education

Classroom teachers have a variety of formative and summative methods they can use to assess students’ performance: homework and class assignments, student feedback and what they say about what and how they are learning, the questions students raise in class, students’ body language, and unit quizzes and tests. Distance teachers can also assess students’ progress, but may need to use different tools and technology than a classroom teacher. Thus, one of the key tasks for distance teachers is to develop ways of obtaining the information they need to conduct assessment of student progress on a regular basis. Collecting this information is part of the learning sequence; it involves determining when, what, and how to test and making instructional choices based on results (Popham, 2011). Teachers in a blended learning class will want to include formative and summative assessments in both the face-to-face and online portions of the class. The following section includes examples of assessment methods and how they can be used in a distance education and/or blending learning environment.

Reviewing Student Online Work

One way for teachers to assess student progress is to regularly review the student’s work and provide feedback. Another option would be using tests and quizzes to assess distance students; this may make distance assessment more parallel to classroom-based assessment. These quizzes could be completed using online websites, posted in a learning management system, or emailed to the student. When providing synchronous remote instruction, teachers can assess students’ work similar to in-person methods, such as asking questions, using real-time formative assessment tools and games, or having students submit writing samples through chat. Since the primary focus of these formative assessments is to gain information to help the teacher in instructional planning, issues about secure testing sites, which are a concern for accountability purposes, are less relevant. 

Most comprehensive online curricula offer some form of tailored assessment (e.g., diagnostic instruments, unit quizzes, or tests) designed to help teachers and students gauge student progress. Teachers can use it to gauge overall understanding of a specific topic as well as to identify specific skills where students may need additional instruction. While these product-tailored assessment measures are not accepted for accountability purposes, they can be valuable tools in monitoring student progress and determining readiness.

Some examples of how teachers review student online work include:


Note, if you are creating your own assessments, do follow some key principles of Universal Design, a framework for developing flexible learning environments or activities that can meet the needs of a wide range of learners. The Center for Applied Special Technology (CAST) provides extensive guidance and resources around Universal Design for Learning, including the National Center on Accessible Educational Materials website.

Be sure items are clear and concise.

Keep things simple so you won’t distract students from the key skills you are trying to assess. Avoid idiomatic language, like “brainstorm ideas” or “think outside the box.” Avoid false cognates—words that sound or look the same but have different meanings in two languages (Dame, 2020). For example, the English verb, “to record”, looks like the Spanish verb “recordar” but  “recordar” means to remember. 

Pay attention to content and language.

Take into account the diversity of the students in your class; consider cultural, linguistic, geographical, gender, disability, or socioeconomic demographic information. Create items based on topics familiar to all students, making sure they are not likely to be viewed as insensitive, biased, or relying unnecessarily on culturally bounded background information (Dame, 2020).

Avoid sensitive topics.

Do not include content involving sensitive or controversial topics that might distract students, like natural disasters, death, crime, or violence. You never know what trauma someone has experienced. If it is essential to include a sensitive topic as the context for an assessment item, let students know ahead of time and give an option to opt out of the item (Dame, 2020).

Culminating Activity

Teachers may also have students work on a culminating activity to show mastery of skill. Some examples of culminating activities include:

In a blended learning scenario, a cohort of students can use online collaboration tools, which will allow you to assess their interactional skills and participation (Herr et al., 2015).

Portfolios

Students and teachers can maintain a portfolio of student work to track and demonstrate progress. Although portfolios do not meet National Reporting System requirements, they can provide additional evaluation information to guide instruction. In a blended learning scenario, integration of portfolios can provide the means to extend classroom-based learning to out-of-class or online work.

Using a Portfolio in Blended Learning

"I teach in a blended Vocational ESL writing class and use Weebly as a digital portfolio for learners. Not only can I easily monitor progress by looking at the Weekly post, but my learners can look back, see their improvement, and use old work to help them with new activities. ."

– an adult ESL teacher in California

 

These portfolios could include:

Using Rubrics for Alternative Assessments

Teachers who use performance-based assessments, like culminating activities or portfolios, provide both clear expectations from the start and incremental feedback along the way. The use of rubrics or assessment tools for sharing assignment expectations, along with offering timely feedback and grading of student work, is central to the effectiveness of student learning.

Analytic Rubric

This common rubric (for a student writing assignment) lists criteria for completion in the left column and evaluation levels across the top. The cells of the grid explain in detail what the teacher will be looking for when they evaluate the work (Rowell, 2020).

 
For example: 

Image of an analytic rubric for writing

(Rowell, 2020)

Holistic Rubric 

This simple rubric is less structured (Gunner, n.d.). A teacher provides a series of letter grades or a range of numbers (1–4 or 1–6, for example) and then assigns expectations for each of those scores. Teachers grade and rate the student’s according to the rubric. This is a faster way to evaluate work but leaves no room for comments or detailed feedback (Rowell, 2020).

For example:  

Image of a holistic rubric

You could use this template to create either an analytic or holistic rubric.

Interaction with Students

Using the telephone or an online tool (such as Skype or Zoom), distance teachers often meet with their students to review their work and ask them questions to assess their understanding of concepts. These meetings may also be held in person for blended students. The following video shows how a K-12 teacher some makes the most of a short conversation by turning it into an interview assessment:

Image preview of a YouTube video
Watch on YouTube https://edtechbooks.org/-uZQu

Watch on YouTube https://edtechbooks.org/-uZQu

Progress Checklists

Skills checklists can show a student’s progress while in the program. Skills checklists may be part of a goal plan or a stand-alone tool used by teachers and students to document skills attainment.

Documenting student progress can support persistence by changing a student’s beliefs about their capabilities and achievements (Drivers of Persistence: Competence, 2013). A visual representation of learned skills can build students’ self-confidence and self-efficacy in terms of their ability to learn and be successful in education. This change in how students view their abilities can have a profound effect on their persistence in the program and achievement. Digital badges, referenced in the previous section, provide a great visual presentation of learner milestones and accomplishments.

Here are some tips for making your own checklist:

Additional Assessment Measures

In addition to the ideas presented above, IDEAL Consortium states have suggested several possibilities for ongoing or interim assessment of distance student progress, including:

Assessment to Meet the NRS Guidelines

The U.S. Department of Education’s Office of Career, Technical, and Adult Education (OCTAE) 2021 National Reporting System (NRS) Technical Assistance Guide states that distance learners can be included in the NRS, as long as states have an approved distance learning policy in their state’s adult education plan. OCTAE first announced this option in 2007, and since then many states and local organizations have included distance learners in their NRS reports. In order to be included in the NRS, distance learners must be assessed according to the same policy that is in place for all adult learners in the state. Your state will provide guidance on how to report distance learners. The following discussion of NRS requirements is intended only to provide some general background information; refer to the appropriate NRS, OCTAE, and state policy documents for specific details.

States must include the following information about assessment in their distance learning policy: 

. The 2021 NRS Technical Assistance Guide states that distance learners may be assessed in person, at a secured proctored program site that meets the state’s assessment policy or via virtual proctoring (remote test administration) when the NRS-approved test publisher allows it. The NRS Technical Assistance Guide states that distance learners “should be post tested after the same amount of instructional time as other students, according to the state’s approved NRS assessment policy” (p. 23). Assessment must be done using a standardized test identified in the state’s assessment policy and must take place in a secure, monitored setting. This does not mean, however, that the assessment must occur at the adult education center. Some adult education organizations have made arrangements with local public schools or libraries and trained staff there to administer and proctor testing for distance learning students living in those communities. A few teachers travel to remote locations to administer the assessments.

Remote test administration that began during the COVID-19 period allows more opportunities for distance learners to be tested. Organizations remotely testing students when in-person contact was not allowed because of COVID-19 have found innovative solutions to this new testing method. See the table below for examples.

 

Remote test administration challenge Possible solutions
Students do not have a device that can be used to take the test. Partner with K-12 school districts to secure permission for adult learners to use their child’s school-issued device for adult basic education activities, such as assessment and online assignments.
Students do not have access to Wi-Fi.

Create a map of local Wi-Fi spots available from places such as libraries and school districts.

Students have parked in the organization’s parking lot to take the test from a car.

More than one student at a time needs to be tested.

Some test publishers allow multiple students to be tested simultaneously.

Have a staff member meet with students to test their technology and set everything up before the student is scheduled for a remote test administration session to make the process more efficient.

Measuring Instructional Time for Distance Learners

Contact Hours

How do you measure instructional time for distance learners? In a classroom, the most commonly used approach is to record “contact hours,” the amount of time a student is physically present in orientation, the classroom, the lab, and so on. This figure determines when a learner becomes an enrolled student (at 12 hours) and when assessment of educational functioning level should be administered (frequently after 40 or 50 hours, but it can be longer). Contact hours can also be counted for distance learners, but these hours extend beyond times when a student is physically present. 

OCTAE’s 2019 NRS Technical Assistance Guide states “contact hours for distance learners can be a combination of actual (face-to-face) contact and contact through telephone, video, teleconference, or online communication, where the participant and program staff can interact and through which participant identity is verifiable” (p. 46). This allows distance education programs to count contact hours for times when a distance teacher provides instruction using the telephone, webinars, video chat technologies, or interaction in the assigned distance learning curriculum.

Proxy Contact Hours

In addition to measuring contact hours, states have the option to report proxy contact hours for distance learners. Proxy contact hours provide an indication of how much instructional time, on average, distance students are likely to spend on specific distance learning activities. From an assessment perspective, proxy contact hours serve the same functions as contact hours: they allow adult education providers to determine when to post-test students. They also provide instructors with another way of monitoring their students’ engagement with the curriculum and help instructors determine where additional support or intervention might be warranted.

Proxy contact hours are assigned using a systematic process. Your state will provide guidance on what proxy contact hours (if any) you will use for your distance learners; this is not typically a decision that individual teachers or adult education centers make. For NRS purposes, the following three models of determining proxy contact hours are acceptable:

States are not required to report proxy contact hours to the NRS. However, if proxy contact hours are reported, they must be used to determine when it is appropriate to post-test students. States that do not use proxy contact hours must provide information in their distance learning policy that explains how they will make decisions about appropriate post-testing intervals.

Posttesting Students

Getting students to come back to the adult education center for post-testing is one of the major challenges facing distance teachers. While remote test administration may resolve transportation issues, other barriers may still exist. Students might not have time to come in or adequate transportation. They might feel unwilling to meet face-to-face due to COVID concerns. Even students who are post-testing remotely might feel reluctant. They may not see the importance of testing, or they may not have a device or adequate space in their home that allows remote testing. Yet post-testing is important both for monitoring student progress to guide instruction and for accountability purposes. 

Posttesting Students

"Our state requires students to return to an adult education class and take a posttest in at least one subject every three months. First, we remind students to go in and take a posttest. We point out how valuable this is to us and them. Then if they do not respond or go in and take a posttest, we block them from class until they go in and take a posttest. If they have a good reason for not post-testing right way, I will give them some extra time."

–A Teacher in Missuori

 

Teachers in IDEAL Consortium states report that they have used the following approaches to encourage post-testing:

Considering Your Assessment Strategies

Activity 6.1 Assessment to Gauge Learner Progress and Guide Instruction

Plan how you will use the different assessment strategies described in this chapter.

Of the strategies listed in this chapter, which will you use and how will you implement them? If you are a practitioner new to distance or blended instruction but working where there is an established program, be sure to first consider what is currently in place.

Activity 6.2 Assessment for NRS Reporting

Articulate how you will fulfill NRS testing and reporting requirements for your distance education program.

You will first need to review your state’s distance education policy and assessment policy. Then, describe how you will handle assessment for NRS reporting of your distance and blended learning students and your plan for posttesting distance students. If you are a practitioner new to distance or blended instruction but working within an established program, be sure to first consider what is currently happening in your distance education program. Note that in the course, IDEAL 101: Foundations of Distance Education and Blended Learning, these prompts are expanded into fully developed collaborative activities for your team to complete together.

Note that in the course, IDEAL 101: Foundations of Distance Education and Blended Learning, these prompts are expanded into fully developed collaborative activities for your team to complete together.

References

Bakerson, M., Trottier, T., & Mansfield, M. (2015). The value of embedded formative assessment: An integral process in online learning environments implemented through advances in technology. In S. Koc, X. Liu, & P. Wachira (Eds.), Assessment in online and blended learning environments (pp. 3–20). Information Age Publishing.

Dame, B., & Lea, K. (2020, May 29). Using universal design to create better assessments. Edutopia. https://www.edutopia.org/article/using-universal-design-create-better-assessments

Edutopia. (2008, July 15)Why is assessment important?  Edutopia. https://edtechbooks.org/-QWe 

Gunner, J. (n.d.). Simple rubric examples for teachers. Your Dictionary. https://examples.yourdictionary.com/simple-rubric-examples-for-teachers.html

Herr, N., Rivas, M., Chang, T., Reveles, J., Tippens, M., Vandergon, V., & Nguyen-Graff, D. (2015). Continuous formative assessment during blended and online instruction using cloud-based collaborative documents. In S. 

Koc, X. Liu, & P. Wachira (Eds.), Assessment in online and blended learning environments (pp. 187–214). Information Age Publishing.

Lauzon, N. (2014, September 10). Checklists and achievement charts. https://www.ldatschool.ca/Checklists-Achievement-Charts

Many, T. W., & Horrell, T. (2014, February). Best practices: Prioritizing the standards using R.E.A.L. criteria. TEPSA News. https://edtechbooks.org/-MXaM 

Miller, A. (2020, April 7). Formative assessment in distance learning. Edutopia. https://www.edutopia.org/article/formative-assessment-distance-learning

Miller, A. (2020, April 28). Summative assessment in distance learning. Edutopia. https://www.edutopia.org/article/summative-assessment-distance-learning

Popham, W. J. (2011). Transformative assessment in action: An inside look at applying process. ACSD. 

Roell, K. (2019, July 3). How to create a rubric in 6 steps. ThoughtCo. https://edtechbooks.org/-isDP 

Sparks, S. D. (2020, April 22). Types of assessments: A head-to-head comparison. Education Week.

U.S. Department of Education. (2021) Technical assistance guide for performance accountability under the Workforce Innovation and Opportunity Act.  U.S. Department of Education, Office of Career, Technical, and Adult Education. https://edtechbooks.org/-Lyr 

World Education. (2013, December 13). Drivers of Persistence: Competence. World Education. https://edtechbooks.org/-QFC

This content is provided to you freely by EdTech Books.

Access it online or download it at https://edtechbooks.org/ideal_dl_handbook/ch6__assessment.