Chapter 3 | Assessing Readiness
Imagine this fairly common scenario for new distance education programs: You decide you want to use distance education to intensify learning for current learners. You might also decide to offer a new complementary online component to your face-to-face classes or provide a learning option for learners who cannot make it to regular class times or are on your program’s waitlist. While all are good reasons to start using distance education, without careful coordination, proactive planning for providing support, and marshaling of resources, the learners who start in this program are not likely to have the support they need to persist.
What can happen is a churn of orientation for new learners, constant follow-up to connect with learners who are not participating, and work to exit learners who have not been regularly participating. Past IDEAL member states all seem to have stories about how this scenario played out and eventually impacted new distance programming. Because resources in adult education are often in short supply, distance education programs have a finite amount of staff time available to support learners. Ideally, this time is used in facilitating students’ learning. In reality, there is sometimes a disproportionate amount of time spent on administration and keeping track of learners. To mitigate the possibility of this happening, programs need to be sure they understand the level and types of support that each learner needs, and have in place plans to provide it. Implicit in this strategy is the need to understand the readiness of potential future learners. This readiness is characterized by learner strengths in several areas, including:
- academic readiness for particular content,
- soft skills or habits of mind (e.g., persistence, time management, and goal setting),
- technology skills, and
- access to a device (e.g., computer, tablet, smartphone) and the Internet.
While the work adult education agencies did to provide emergency remote instruction during the COVID-19 pandemic has shown that learners of all levels can participate in distance learning, it may be worthwhile determining which learners are best served with a distance education model at your agency, what skills and access are necessary to participate, and how you can set these learners up for success.
If you are working to provide distance education opportunities for a wide range of learners, you can learn from past and current practices to assess student readiness and determine which learning materials best align with their competencies and needs. Then you are better equipped to provide the supports necessary for learners at all levels to boost persistence in educational opportunities.
The Importance of Assessing Readiness
Once we had to move all of our services online, we quickly realized the importance of assessing our learners’ readiness for online learning. Staff helped learners to determine what access they had to technology and enrolled learners in our device loaning program if they didn’t have access to technology at their home. We also assessed their digital literacy skills and made sure that learners left knowing how to use the technology we were providing. We also covered soft skills needed for online learning during our “Digital Boot Camp.” This helped to set learners up for success with online learning.
– An adult education teacher in California
Alignment of Learner Knowledge with Proposed Curriculum
It is important to determine the skills a learner brings to the learning experience (e.g., reading proficiency and computer competencies). First, this requires that instructors be familiar with the objectives of a course and the skills and competencies needed to engage with the curriculum and instructional materials. Second, teachers need to examine a student’s academic skills and knowledge, which can be done with a formal assessment tool (e.g., TABE, CASAS, or BEST), customized placement assessments, and/or by informal means (e.g., observing the ease with which they read materials about the program and listening to their oral English skills as they talk to the teacher). Seminole State College has created this oral assessment to help with determining placement and learning needs for their ESOL students.
Teachers working in a blended learning environment, who see learners in class, will likely have an understanding of their learners’ academic readiness for the online activities needed to do coursework. Teachers supporting students working completely remotely and independently need to be sure students have the academic skills needed to handle the work. Assessing students prior to instruction helps ensure the program is a good fit for students’ needs and abilities.
We created academic placement tests based on the objectives of our program curricula. We do the whole thing using the telephone, WhatsApp, and Zoom. Starting with an intake survey, we then move to an oral placement. If they are at a high enough level of English language proficiency, they also get a reading placement. The same staff person does all of the assessment.
—Carlos Rosario International Public Charter School in Washington, DC explains how they assess learner competencies at a distance
Most organizations already have a system in place for assessing new students, but current assessments should be expanded to measure a student’s capacity to use technologies—either in class or in online independent work. Some organizations require a particular assessment tool. The more closely placement assessments match the curricular content and skills required to access learning, the more useful the process will be. (For more information about assessment and adult education distance learners, see the original Project IDEAL Working Paper 1, Assessment and Accountability Issues in Distance Education for Adult Learners. Although published in 2002, it still has relevance today.)
Assessment of Nonacademic Competencies
Learner persistence and success in distance education depends on more than students’ academic skills and knowledge. Distance and blended learning require that students be able to organize their time, work independently, have good study skills, and solve problems using technology. Learners who lack these skills can come against barriers in a distance program. These noncognitive skills become very important in distance education, where students are not enrolled in an onsite classroom-based course, and teachers may meet with their students only once or twice over an entire course, with the remainder of the communication occurring via telephone, email, online learning features, or videoconference.
Additionally, depending on the distance learning model used, distance students may have little or no face-to-face contact with other students taking the same course. This means distance students need to possess the characteristics (e.g., independence, self-motivation, and organization and study skills) that enable them to succeed without the extra support a classroom environment typically provides. Thus, early in program orientation or the assessment process, teachers should find some way to assess such competencies. There are many ways to assess these characteristics, ranging from questionnaires to informal interviews with potential students.
Habits of Mind and Skills That Matter
Habits of Mind have been defined as the behaviors required to support learning and successful application of the knowledge that students already possess. Costa and Kallick (2000) list the following characteristics of Habits of Mind:
These habits come into play when a learner faces a challenge or needs to solve a problem. Such events require a learner to creatively draw on prior knowledge and not give up. Many of these habits are encompassed in the Teaching Skills That Matter in Adult Education project of the U.S. Department of Education, Office of Career, Technical, and Adult Education. These are the transferable skills required for success in daily life, work, and school.
- Adaptability and willingness to learn
- Critical thinking
- Interpersonal skills
- Navigating systems
- Problem solving
- Processing and analyzing information
- Respecting differences and diversity
The Habits of Mind Self-Assessment Rubric created by the Institute for Habits of Mind provides a means to informally gauge these soft skills and can be used as a guide to help teachers and learners together determine readiness for independent work.
In addition to the assessments described above, there are several online self-assessment surveys that help students determine whether learning independently online (in either distance or blended models) will work for them.
Sample Intake Survey: Appendix A of this handbook is a questionnaire developed by IDEAL Consortium leadership and informed by past member observations about questions required for intake. Students can take the survey alongside the facilitator in an orientation session.
YWCA National Capital Area Learner Readiness Survey: This short survey was developed in Google Forms specifically for intake in adult basic skills programs. It covers a range of readiness areas, including study environment, time available for distance learning, access to devices and the internet, and how students problem-solve.
YWCA National Capital Area Motivation Inventory: This short survey may help you understand a learner’s current motivation and commitment to working independently. You could use the survey results as the basis for a conversation during an intake session.
MNSCU Distance Learning Quiz: The Minnesota State Colleges and Universities system offers an online education readiness quiz covering motivation, learning preferences, time management, commitment, academic readiness, and technology skills/computer access.
Penn State Self-Assessment: This brief quiz asks questions about time management, study skills, personal organization, and technical skills. The quiz offers feedback that teachers can use as the basis of a conversation about readiness.
Questionnaires of this type provide another method for determining the most appropriate educational plan for students. Concrete information about time usage, study skills, and the ability to organize is a valuable component of orientation for distance and blended learning students. We encourage you to explore the resources above, consider the requirements of your distance or blended program, and then create your own.
Google Forms and Survey Monkey are both useful tools for gathering, organizing, and storing information. If your organization has Adobe Acrobat Pro, you can use that to create forms that automatically transfer gathered information to a response file.
Digital Literacy Skills
Foundational computer, telephone, and mobile device skills (e.g., proficiency with common computer applications, Internet browsers, and use of email) are a necessity for students studying online. It is also critical that learners have a basic understanding of how websites and hyperlinking work. While students know to turn the page of a book to find what comes next, they might not know that they need to scroll down on a web page to see all of the information or follow an important hyperlink to needed information. Computer knowledge needed to study online includes skills such as:
- Using the mouse to navigate on the screen and to click on appropriate items
- Using a keyboard to enter text. While touch-typing is not essential, the student needs to have a level of comfort using the keyboard to enter responses and complete assignments
- Being able to connect—and stay connected—to the Internet
- Navigating web pages, including using the back button and managing new tabs in browser windows
- Composing and replying to texts and emails
- Logging in to programs
- Retrieving passwords, uploading files
Students who are participating in a program using a mobile device may also require some additional skills, such as downloading and installing apps. The EdTech Center@World Education created a digital literacy self-assessment tool that can be adapted to meet the needs of your learners.
The Voice of Experience
Students entering into a DL program with our institution are asked to spend a minimum of 8 hours in the computer lab. This allows for the student and teacher to get to know one another, it allows for the student to become acquainted with the computer to be used in a supervised atmosphere, and it allows for students to understand what is expected of them, what their place is in their education and their goal attainment. In addition, since distance learning requires that students have good reading and organizational skills, there is a questionnaire that students take to see if they will be successful in said program.
– a distance education teacher in Arizona
Some sites have opted to observe students’ computer use at an orientation as an informal assessment of their computer skills. It may be helpful to develop a quick checklist to assess students’ computer skills. If you are working remotely, you may need to do this from a distance. One strategy is to ask students who express interest in distance education programs to respond to an email containing an attachment that students must open, fill out, and return. Northern Shenandoah Valley Adult Education in Virginia has used a Facebook Messenger greeting to engage potential learners who land on their site. The greeting has prompts to help students indicate their interest and questions. Students who can successfully respond to this usually have the needed computer skills to take a distance course.
Some organizations participating in the IDEAL Consortium design their distance learning orientations to include an extended period of time for the student to explore the online curriculum. Several organizations have the student complete an entire online lesson during the orientation session. This allows the teacher and students an opportunity to determine if students have the requisite skills to use the online program. It also gives students a chance to decide if they are comfortable with this educational approach, whether they possess the range of digital literacy required (both basic computer skills and higher level skills, like using technology to solve problems and information literacy).
Many adult education programs and libraries across the country use the Northstar Digital Literacy Assessment to understand learner competency with basic computer skills, Internet, email, computer operating systems, Microsoft Office Suite software, social media, and information literacy. This popular and free digital literacy assessment was developed specifically for use with adult learners. The standards on which the assessment modules are based were developed by librarians and adult education and workforce development practitioners. Each of the 14 available assessments takes about 30 minutes to complete. Programs could choose which assessments are most relevant to their learners’ goals and the distance education program.
For students who need additional skills prior to beginning the distance education program, or help along the way, the organization may choose to provide training (for example, running a one- or two-session class on basic computer skills to help them get started). You may wish to do an analysis of the online materials that are used in your distance and blended learning and then focus training on the skills needed for student success and persistence. Some popular and free learning sites are GCFLearnFree, the Public Library Association’s DigitalLearn.org, Google’s Applied Digital Skills curriculum for students, or this computer basics module from Northstar Digital Literacy Project.
Computer and Internet Access
In a classroom setting, educational materials and technology are sometimes made available to the students (e.g., computer labs, tablets, and the Internet). Organizations are also likely to employ someone who is knowledgeable in those technologies and who can help teachers and students best utilize them. Students who cannot come into the organization to use these resources may not have access to the same breadth of technology and support. Though computer and Internet access among these adults is increasing at a very rapid rate, organizations must problem-solve ways to provide students with access to all of the materials and technologies they will need to get the most from their distance studies.
Some organizations have solved technology and distribution problems by providing open computer lab time where distance and blended learners can work online. Others have made arrangements with local libraries, public schools, community-based organizations, and One-Stops to allow use of their computer labs. In Rhode Island, the RI Family Literacy Initiative (RIFLI) lends tablets and mobile hotspots to enrolled learners who do not have home access. If you do set up a lending program, you will likely need to set up technology lending agreements with your learners. The Dover Adult Learning Center Laptop Loan Agreement is an excellent example of what needs to be included.
There are also nationwide programs that support home broadband connections. Everyone On is a nonprofit expanding access to high-speed, low-cost Internet service and refurbished computers by partnering with local Internet service providers, municipalities, and local nonprofit organizations. A range of broadband options are available at a fraction of their usual cost to families with school-age children who qualify for free or reduced cost lunch. Similarly, the Federal Communication Commission’s Affordable Connectivity Program provides discounts on internet access and device purchases. The program provides a discount on monthly service of $30 per month for eligible low-income households. Subscribers can use the benefit to purchase wired or wireless services from participating broadband providers.
The need for access to digital devices and the internet was brought into stark relief as programs shut down around the country because of the COVID-19 pandemic. Indeed, a survey of nearly 800 program administrators and instructors across the United States showed that digital access was the main barrier to participation in learning and that programs that had already put into place processes and resources for loaning devices and Internet access were those able to continue supporting learners without resorting to paper packets (Belzer et al., 2020). The following are some promising initiatives to address digital exclusion issues in the United States.
- The National Cristina Foundation has launched a nationwide call for surplus computers from corporate or governmental sources. It then matches donors with nearby refurbishers, who in turn prepare and distribute the equipment at low or no cost to organizations in need.
- Tech Goes Home (TGH) is a nonprofit with initiatives in five cities that provides training to help learners of all ages use the internet and computers. In TGH cities such as Chattanooga, TN, participants who complete a 15-hour digital skills training are offered an extremely low cost laptop. TGH also provides directories, localized curriculum, and guides to common digital tools and resources.
- Organizations that are part of the Wash & Learn Initiative, like Libraries Without Borders and the Laundry Literacy Coalition, provide spaces for internet access and learning digital skills in nontraditional locations in the community—such as laundromats.
Many programs also began employing digital navigators who provided a comprehensive approach to ensuring learners had both digital access and the digital literacy skills needed to use the devices. Digital navigators were sometimes teachers and other times dedicated staff or volunteers who focused on digital equity and literacy. The Digital U.S Digital Navigator Resources website features tools that digital navigators or other practitioners can use to meet the needs of learners.
To get a sense of your learners’ technology access needs, consider adding a self-assessment that asks about access, skills, and comfort. This Distance Learning Technology Access Survey from the YWCA National Capital Area can be delivered over a mobile device.
Defining Learner Readiness
Activity 3.1 Screening and Learner Readiness Checklist
Describe how you will measure a range of readiness characteristics and then how you will respond if learners require further preparation to succeed in online learning.
Consider the needs of your learners, resources available, and administrative processes at your organization. Then develop a list of readiness characteristics that you will use to determine the supports needed for learners to successfully participate in your distance or blended learning opportunities.
Note that in the course, IDEAL 101: Foundations of Distance Education and Blended Learning, these prompts are expanded into fully developed collaborative activities for your team to complete together.
Belzer, A., Leon, T., Patterson, M., Rhodes, C., Salas-Isnardi, F., Vanek, J., Webb, C., Willson-Toso, B. (2020). COVID-19 rapid response report from the field. Open Door Collective, EdTech Center@World Education, ProLiteracy. https://edtechbooks.org/-PEzK
Building a digitally resilient workforce: Creating on-ramps to opportunity. (2020). World Education, Inc., and the Digital US Coalition. https://edtechbooks.org/-xnS
Costa, A. L., Kallick, B., & Anderson, J. (2008). Part 1: Discovering and exploring habits of mind. A Developmental Series, Book 1. In A. L. Costa & B. Kallick (Eds.), Learning and leading with habits of mind: 16 essential characteristics for success. Alexandria, VA: Association for Supervision and Curriculum Development. https://edtechbooks.org/-ZSNJ
U.S. Department of Education. Teaching skills that matter in adult education. (2019) LINCS | Adult Education
and Literacy | U.S. Department of Education.
CC BY-NC-SA: This work is released under a CC BY-NC-SA license, which means that you are free to do with it as you please as long as you (1) properly attribute it, (2) do not use it for commercial gain, and (3) share any subsequent works under the same or a similar license.
End-of-Chapter Survey: How would you rate the overall quality of this chapter?
- Very Low Quality
- Low Quality
- Moderate Quality
- High Quality
- Very High Quality