E is for Evaluate

Evaluation is a measurement of the training program, not an assessment of the students.

Evaluation is something we do during the creation of learning materials, known as Formative Evaluation, but also after learning programs have been generated and running, known as summative evaluation. Summative Evaluation is sometimes also known as Program Evaluation, and there are entire fields of professional that do program evaluation.

A completed program evaluation may have been what identified the need to develop training in the first place. There can be overlap between program evaluation and initial goal analysis. Program Evaluation reports can be an input in to the “analysis” phase of design.

Formative Evaluation

Formative evaluations happen throughout the design process. For example,  peer reviews of the design documents. 

In online and eLearning, we may have a sample student or two run through the navigation of the course or a specific lesson or activity to collect feedback and improve the design.

A more formal way of doing formative evaluation is to run a pilot offering of the course. At this point, you expect that you will make modifications to the course based upon student feedback. Sometimes you hold a focus group immediately after the course to get feedback while it is still fresh. Usually this is done with the designers, or some third party, not the instructors.

To save time and money, some organizations double up and use the train-the-trainer, or (T3) session as the course pilot. Trainers then provided feedback that is used to update the course before it goes into larger distribution.

Summative Evaluation

The process for summative evaluation is typically more formal than formative evaluation. The basic process involves 4 steps:

  1. Define the goal of the evaluation and specify the specific areas that will be evaluated. In addition to student learning, goals for evaluation may include costs, often termed as RoI, or Return on Investment. Training can be expensive and organizations want to know it was worthwhile.
  2. Collect data to support your evaluation. This may include secondary data, that is data that already exists, or primary data, that you collect specifically for the evaluation. Secondary data may include aggregated student assessment results. Not that in this case, the student assessments were done to support student learning, where in evaluation they are used to evaluate the program as a whole. Primary data collection may include interviews and surveys created just for the purpose of evaluating the training program.
  3. Analyze the data. The evaluator looks at the goal of the  evaluation and the data collected to see how well the goal was achieved.
  4. Communicate the results of the evaluation. This is often done in the form of an evaluation report.
Evaluation in ID

This content is provided to you freely by EdTech Books.

Access it online or download it at https://edtechbooks.org/demystifyingID/e_is_for_evaluate.