Assessing Readiness
Determining What Supports Students Need to Succeed in Distance and Digital Education
Introduction
There are several reasons for starting to offer a range of new distance and digital education programs. You might want to use a hybrid approach to intensify learning for current learners. You might also decide to offer a new complementary online component to your in-person class for blended learning or create a HyFlex class to provide a learning option for learners who cannot make it to regular class times. While all are good reasons to start using distance and digital education, without careful coordination and proactive planning for providing support, you may not have the resources to provide all learners the support they need to persist.
What can happen is a churn of orientation for new learners, constant follow-up to connect with learners who are not participating, and, if it’s a distance education course, extra work to exit learners who have not been regularly participating. Past IDEAL member states all seem to have stories about how this scenario played out and eventually impacted new distance programming. Because resources in adult education are often in short supply, distance education programs have a finite amount of staff time available to support learners. Ideally, this time is used to facilitate learners’ learning. In reality, there is sometimes a disproportionate amount of time spent on administration and keeping track of learners. To mitigate this, programs need to be sure they understand the level and types of support that each learner needs and the plans to provide it. Implicit in this strategy is the need to understand the readiness of potential future learners. This readiness is characterized by learner strengths in several areas, including:
academic readiness for particular content,
soft skills or habits of mind (e.g., persistence, time management, goal setting),
technology skills
- access to a device (e.g., computer, tablet, smartphone) and the internet.
While the work adult education agencies did to provide emergency remote instruction during the COVID-19 pandemic showed us that learners of all levels can participate in distance and digital learning, it is worthwhile to determine which learners are best served by the different distance and digital education models offered at your organization, what skills and access learners need in order to participate, and how you can set these learners up for success. It is helpful to learn from past and current practices your organization uses to assess learner readiness and expand upon them to help you determine which learning modalities and materials best align with learners’ competencies and needs. Then, you are better equipped to provide the supports to boost persistence in educational opportunities for learners.
The Importance of Assessing Readiness
"Once we had to move all of our services online, we quickly realized the importance of assessing our learners’ readiness for online learning. Staff helped learners to determine what access they had to technology and enrolled learners in our device loaning program if they didn’t have access to technology at their home. We also assessed their digital literacy skills and made sure that learners left knowing how to use the technology we were providing. We also covered soft skills needed for online learning during our 'Digital Boot Camp.' This helped to set learners up for success with online learning."
– An adult education teacher in California
Alignment of Learner Knowledge with Proposed Curriculum
It is important to determine the skills a learner brings to the learning experience (e.g., reading proficiency, digital skills, positive experiences with technology use). First, this requires that instructors be familiar with the objectives of a course and the skills and competencies needed to engage with the curriculum and instructional materials. Second, teachers need to examine a learner’s academic skills and knowledge, which can be done with a formal assessment tool (e.g., TABE, CASAS, BEST), customized placement assessments, and/or by informal means (e.g., observing the ease with which they read materials about the program and listening to their oral English skills as they talk to the teacher). Seminole State College has created this oral assessment to help with determining placement and learning needs for their ESOL learners.
Teachers who deliver instruction for blended, HyFlex, or hybrid learning and see learners in class, will likely have an understanding of their learners’ academic readiness for the online activities needed to do coursework. Teachers supporting learners working completely remotely and independently need to be sure learners have the academic skills needed to handle the work. Assessing learners prior to instruction helps ensure the program is a good fit for learners’ needs and abilities.
Aligning to Your Learners
"We created academic placement tests based on the objectives of our program curricula. We do the whole thing using the telephone, WhatsApp, and Zoom. Starting with an intake survey, we then move to an oral placement. If they are at a high enough level of English language proficiency, they also get a reading placement. The same staff person does all of the assessment."
– Carlos Rosario International Public Charter School in Washington, DC explains how they assess learner competencies at a distance
Most organizations already have a system in place for assessing new learners, but current assessments should be expanded to measure a learner’s capacity to use technologies—either in class or in online independent work. Some organizations require a particular assessment tool. The more closely placement assessments match the curricular content and skills required to access learning, the more useful the process will be. (For more information about assessment and adult education distance learners, see the original Project IDEAL Working Paper No. 1: Assessment and Accountability Issues in Distance Education for Adult Learners (Young et al., 2002). Although published over 20 years ago, it still has relevance today.)
Assessment of Nonacademic Competencies
Learner persistence and success in distance and digital education depends on more than learners’ academic skills and knowledge. Distance, blended, hybrid, and HyFlex learning require that learners are able to organize their time, work independently, have good study skills, and solve problems using technology. These nonacademic skills become very important in distance education, where students are not enrolled in an onsite classroom-based course, and teachers may meet with their students only once or twice over an entire course, with the remainder of the communication occurring via telephone, email, online learning features, or videoconference.
Additionally, depending on the distance and digital education modality used, distance students may have little or no in person contact with other students taking the same course. This means distance students need to possess the characteristics (e.g., independence, self-motivation, organization and study skills) that enable them to succeed without the extra support a classroom environment typically provides. Thus, early in program orientation or the assessment process, teachers should find some way to assess such competencies. There are many ways to assess these characteristics, ranging from questionnaires to informal interviews with potential students.
Habits of Mind and Skills That Matter
Habits of Mind have been defined as the behaviors required to support learning and successful application of the knowledge that students already possess. Costa (2008) lists the following characteristics of Habits of Mind:
|
|
The Habits of Mind Self-Assessment Rubric created by the Institute for Habits of Mind (n.d.) provides a means to informally gauge these soft skills and can be used as a guide to help teachers and learners together determine readiness for independent work.
These habits come into play when a learner faces a challenge or needs to solve a problem. Such events require a learner to creatively draw on prior knowledge and not give up. Many of these habits are encompassed in the Teaching Skills That Matter in Adult Education, a project of the U.S. Department of Education, Office of Career, Technical, and Adult Education (OCTAE). These are the transferable skills required for success in daily life, work, and school.
Adaptability and willingness to learn
Communication
Critical thinking
Interpersonal skills
Navigating systems
Problem solving
Processing and analyzing information
Respecting differences
Self-awareness
The free application SkillBlox offers digital playlists of resources that are tagged with these skills. Teachers can use SkillBlox to locate resources to bring into their instruction, or create a “Blox” and assign it directly to students. World Education’s Personal & Workplace Success Skills Library offers a wealth of information about teaching and assessing such skills. The library is a collection of resources to guide the integration of personal and workplace success skills into curriculum and instruction, advising and coaching, assessment, and program design. These resources were selected for adult education, higher education, workforce development, and career and technical education programs serving adult and older youth learners and workers, including English language learners.
Other Assessments
In addition to the assessments described above, there are several online self-assessment surveys that help students determine whether learning independently online (in either distance or blended models) will work for them.
Sample Intake Survey: Appendix D of this handbook is a questionnaire developed by IDEAL Consortium leadership and informed by past member observations about questions required for intake. Students can take the survey alongside the facilitator in an orientation session.
YWCA National Capital Area Learner Readiness Survey: This short survey was developed in Google Forms specifically for intake in adult basic skills programs. It covers a range of readiness areas, including study environment, time available for distance learning, access to devices and the internet, and how students problem-solve.
YWCA National Capital Area Motivation Inventory: This short survey may help you understand a learner’s current motivation and commitment to working independently. You could use the survey results as the basis for a conversation during an intake session.
Penn State Self-Assessment: This brief quiz asks questions about time management, study skills, personal organization, and technical skills. The quiz offers feedback that teachers can use as the basis of a conversation about readiness.
Questionnaires of this type provide another method for determining the most appropriate educational plan for students. Concrete information about time usage, study skills, and the ability to organize is a valuable component of orientation for distance and blended learning students. We encourage you to explore the resources above, consider the requirements of your distance or blended program, and then create your own.
Google Forms and Survey Monkey are both useful tools for gathering, organizing, and storing information. If your organization has Adobe Acrobat Pro, you can use that to create forms that automatically transfer gathered information to a response file.
Digital Literacy Skills
Foundational computer and mobile device skills (e.g., proficiency with common applications, Internet browsers, texting, and use of email) are a necessity for students studying online. It is also critical that learners have a basic understanding of how websites and hyperlinking work. While students know to turn the page of a book to find what comes next, they might not know that they need to scroll down on a web page to see all of the information or follow an important hyperlink to essential information.
Assessing Digital Skills
Some sort of digital skills assessment is useful to help teachers understand a student’s digital skills gaps—not to screen them out of digital education but to understand what tasks they are ready for and what skills they need to build.
Checklists and Surveys
Some sites have opted to observe students’ computer use at an orientation as an informal assessment of their digital literacy. It may be helpful to develop a quick checklist to assess learners’’ digital skills, for example this BRIDGES Digital Skills Checklist, which lists the digital skills needed in distance learning. Briya Public Charter School in Washington, DC, created another excellent example based on observing the skills that learners need to succeed in remote instruction. Alternatively, you could use a student-friendly digital skills and access survey, which can be customized to align with the skills and devices you know you would like your learners to possess.
These resources are examples of informal assessment that help teachers and learners to determine if learners have the necessary digital skills to use the online program. They also give learners a chance to decide if they are comfortable with this educational approach and whether they possess the range of digital literacy required (both basic computer skills and higher level skills, like using technology to solve problems and information literacy).
The Voice of Experience
"Students entering into a DL program with our institution are asked to spend a minimum of 8 hours in the computer lab. This allows for the student and teacher to get to know one another, it allows for the student to become acquainted with the computer to be used in a supervised atmosphere, and it allows for students to understand what is expected of them, what their place is in their education and their goal attainment. In addition, since distance learning requires that students have good reading and organizational skills, there is a questionnaire that students take to see if they will be successful in said program."
– A distance education teacher in Arizona
A more comprehensive list of skills is outlined within the BRIDGES Digital Skills Framework. This framework is designed to be a 'bridge' for learners who have had limited access to technologies and may struggle to accomplish common tasks in work, schooling, and daily life. The BRIDGES Digital Skills Framework includes 75 skills across 10 domains organized into these three overarching categories and subtopics:
Gateway Skills: foundational skills required to use a device and participate online.
Mobile: Understanding basic functions of a mobile device to communicate and access goods and services.
Device Ownership: Practices that support device longevity, including physical care, protective software, and using technical support.
Privacy and Security: Maintenance of practices to secure digital identity, recognize threats, and understand the broader safety implications of working in a digital environment.
Productivity Skills: Skills needed to leverage technology to communicate, create, and share content in personal, educational, and professional contexts.
Communication: Exchanging information with others on digital platforms using various strategies to collaborate, share, and communicate.
Creation: Engaging in digital spaces to design, create, and revise content online.
- Workplace and Productivity: Advancing workplace success and professionalism through engagement with an organization's online tools and other supportive digital systems.
Independent Learning Skills: Skills to support finding information, performing everyday tasks, and participating in continuous learning within an increasingly digital world.
Information Skills: Skills to apply, evaluate, and manage information across digital and physical environments.
Lifelong Learning: Engagement in self-assessment of digital skills. Using self-reflection to tailor accessible digital environments and continue learning.
- Online Life: Access to online resources that support digitalization of daily tasks and socialization within a broader digital community.
The framework is part of the broader BRIDGES Digital Resilience Toolkit, which includes a variety of checklist templates that can be adapted to self-assess and monitor learner progress around sets of skills relevant to common goals. These checklists are organized by a variety of user types, as well as based on the specific goals of individual users, so they can be used and adapted by teachers and learners to self-assess and to monitor learners’ skill strengths and gaps. BRIDGES also includes instructional resources and supporting guidance to help teachers shape digital skills instruction that will help learners build digital literacy.
Standardized Assessment of Digital Skills
Many adult education programs and libraries across the country use the Northstar Digital Literacy Assessment to understand learner competency with essential computer skills, essential software skills, and technology use in daily life. This popular and free digital literacy assessment was developed specifically for use with adult learners. The standards on which the assessment modules are based were developed by librarians and adult education and workforce development practitioners. Each of the available assessments takes about 30 minutes to complete. Programs could choose which assessments are most relevant to their learners’ goals and the distance education program.
For students who need additional skills prior to beginning the distance education program, or help along the way, the organization may choose to provide training (for example, running a one- or two-session class on basic computer skills to help them get started). You may wish to do an analysis of the online materials that are used in your distance and digital education and then focus training on the skills needed for student success and persistence. Some popular and free learning sites are GCFGlobal, the Public Library Association’s DigitalLearn.org, Google’s Applied Digital Skills curriculum, or this computer basics module from Northstar Digital Literacy. These and other resources are all included in the Digital Skills Library, part of the BRIDGES Digital Resilience Toolkit.
Computer and Internet Access
In a classroom setting, educational materials and technology are sometimes made available to the students (e.g., computer labs, tablets, and the internet). Organizations are also likely to employ someone who is knowledgeable in those technologies and who can help teachers and students use them. Students who cannot come into the organization to use these resources may not have access to the same breadth of technology and support. Though computer and internet access among these adults is increasing at a very rapid rate, organizations must problem-solve to provide students with access to all of the materials and technologies they will need to get the most from their distance studies.
.png)
Some organizations have solved technology and distribution problems by providing open computer lab time where distance and blended learners can work online. Others have made arrangements with local libraries, public schools, community-based organizations, and One-Stops to allow use of their computer labs. In Rhode Island, the Rhode Island Family Literacy Initiative (RIFLI) lends tablets and mobile hotspots to enrolled learners who do not have home access. If you create a lending program, you will likely need to set up technology lending agreements with your learners. The Dover Adult Learning Center Laptop Loan Agreement is an excellent example of what needs to be included.
There are also programs that support national digital access efforts and low(er) cost home broadband connections. EveryoneOn has a tool to help identify local offers. Lifeline Support is a benefit program to help households purchase broadband and devices.
The need for access to digital devices and the internet was brought into stark relief as programs shut down around the country because of the COVID-19 pandemic. Indeed, a survey of nearly 800 program administrators and instructors across the United States showed that digital access was the main barrier to participation in learning. Programs that had already put into place processes and resources for loaning devices and internet access were able to continue supporting learners without resorting to paper packets (Belzer et al., 2020). The following are some promising initiatives to address digital exclusion issues in the United States.
- The National Cristina Foundation has launched a nationwide call for surplus computers from corporate or governmental sources. It then matches donors with nearby refurbishers, who in turn prepare and distribute the equipment at low or no cost to organizations in need.
- Tech Goes Home (TGH) is a nonprofit with initiatives in five cities that provides training to help learners of all ages use the internet and computers. In TGH cities such as Chattanooga, TN, participants who complete a 15-hour digital skills training are offered an extremely low-cost laptop. TGH also provides directories, localized curriculum, and guides to common digital tools and resources.
Many programs also began employing Digital Navigators who provide a comprehensive approach to ensuring learners have both digital access and the digital literacy skills needed to use the devices. Digital Navigators are dedicated staff or volunteers who focus on help people use digital technologies. The Digital US Digital Navigator Resources website features tools that Digital Navigators or other practitioners can use to meet the needs of learners. More information on Digital Navigators and digital navigation services can be found in Chapter 4.
To get a sense of your learners’ technology access and digital literacy and digital literacy needs, consider adding a self-assessment that asks about access, skills, and comfort. This Distance Learning Technology Access Survey from the YWCA National Capital Area can be delivered over a mobile device. You might print the survey for learners who don’t have access to a digital device. One agency in California worked with their proctors to administer paper versions of the Northstar Digital Literacy Screener during Comprehensive Adult Student Assessment Systems (CASAS) assessments to help teachers understand their students’ digital literacy levels and help the administration plan its schoolwide digital literacy strategy.
Defining Learner Readiness
Activity 3.1 Screening and Learner Readiness Checklist
Describe how you will measure a range of readiness characteristics and respond if learners require further preparation to succeed in online learning.
Consider the needs of your learners, resources available, and administrative processes at your organization. Then develop a list of readiness characteristics that you will use to determine the supports needed for learners to successfully participate in your distance and/and/or digital learning opportunities.
Note that in the course, IDEAL 101: Foundations of Distance and Digital Education Foundations of Distance and Digital Education, these prompts are expanded into fully developed collaborative activities for your team to complete together.
Suggested Resources for Further Exploration
Please see Appendix A for a list of useful resources related to this topic that you may want to explore more.
References
Belzer, A., Leon, T., Patterson, M., Rhodes, C., Salas-Isnardi, F., Vanek, J., Webb, C., & Wilson-Toso, B. (2020). COVID-19 rapid response report from the field. ProLiteracy. https://www.literacynewyork.org/documents/Covid-19/COVID-19-Report.pdf
Costa, A. L. (2008). Describing the habits of mind. In A. L. Costa & B. Kallick (Eds.), Learning and Leading with Habits of Mind : 16 Essential Characteristics for Success (pp. 15–41). Association for Supervision and Curriculum Development.
Institute for Habits of Mind. (n.d.). Habits of mind self-assessment rubric category exemplary habits proficient habits apprentice habits beginning habits. https://www.habitsofmindinstitute.org/wp-content/uploads/2016/11/S8.5-Part4-Habits-of-Mind-self-assessment.pdf
LINCS. (n.d.). Teaching skills that matter in adult education. U.S. Department of Education. https://lincs.ed.gov/state-resources/federal-initiatives/teaching-skills-matter-adult-education
Young, S. J., Johnston, J., & Hapgood, S. E. (2002). Assessment and accountability issues in distance education for adult learners. University of Michigan. https://worlded.me/AssessmentandAccountabilityIssues