For the past three decades, my colleagues and I have been tracking down evidence-based principles for the design of multimedia instruction, that is, instruction involving words and graphics (Mayer, 2021, 2022). Our work has produced over 200 experimental comparisons, yielded a set of 15 evidence-based principles for multimedia instructional design (along with boundary conditions for their applicability), and contributed the cognitive theory of multimedia learning, which is grounded in an understanding of how the human mind works. As with any piece of active scientific inquiry, this effort has evolved and expanded over the years thanks to the efforts of a cohort of researchers from around the world as represented in the most recent edition of The Cambridge Handbook of Multimedia Learning (Mayer & Fiorella, 2022).
In his recent chapter, "The Case for Rethinking Multimedia," Hinderliter (2021, p. 57) first complements our work: "Mayer's Cognitive Theory of Multimedia Learning (CTML) has been a mainstay of instructional design for nearly two decades." Then, he immediately diminishes our work: "but recent developments in theoretical frameworks relevant to multimedia learning point to a reversal of many previously held assumptions." In his efforts to build a case against our work, Hinderliter (p. 57) seeks to "examine multiple issues regarding CTML that question its influential status within the instructional design community." The goal of this essay is to systematically evaluate each of five areas of concern raised in Hinderliter's chapter. I close with an examination of what I consider to be some areas for improvement in our field's quest for evidence-based principles of multimedia instructional design.
Criticism 1: The Research Base Is Limited to 60 Experiments
Hinderliter's first unwarranted claim is that our proposed instructional design principles are based on "sixty previous tests conducted within 20 studies." Hinderliter points out that some studies are limited in terms of the small number of participants, use of narration rather than printed text, or reliance on system-based timing of presentations.
Instead of the 60 experimental tests that Hinderliter attributes to us, our latest edition of Multimedia Learning (Mayer, 2021) includes over 200 experimental tests that we have conducted covering a variety of instructional venues. In a field where claims are often based on a single study, we have maintained a commitment to replicating an effect multiple times before we offer it as a potential instructional design principle. In short, we agree that design principles should be backed up by a body of rigorous experimental tests, published in peer-reviewed journals. With over 200 published experimental tests, our lab is contributing to that worthy goal. I conclude that Hinderliter's first criticism is unwarranted.
Criticism 2: The Design Principles Are Based Solely on Outdated Technologies
Hinterliter castigates CTML for being based on research conducted during "the last decade of the twentieth century" (p. 63). He falsely claims that all of the studies "offered in support of CTML include paper-based treatments...as well as computer-based treatments involving HyperCard stacks on a monochrome Macintosh IIci computer" and that "few previous studies have been thoroughly replicated using adequate sample size within modern online instructional environments" (p. 63). Hinterliter correctly notes that today's students are "highly acclimated to the use of animation, video, and narration," but suggests that our work does not involve these media.
Hinterliter appears to be basing his critique on a review published more than 20 years ago (Mayer, 2002), and he does not cite any of our research studies published since then. He ignores current reviews of our work (e.g., Mayer, 2021) that are based on more than 200 experimental tests including educational technologies such as narrated animation, instructional video, interactive simulations and games, animated pedagogical agents, and immersive virtual reality. Most studies have been replicated numerous times. I conclude that Hinterliter's second claim is unwarranted.
Criticism 3: The Results Do Not Replicate
In perhaps his most destructive claim, Hinterliter states: "The landmark results of early research into cognitive learning principles have proven difficult to replicate" (p. 63). In spite of making a serious claim against the findings of our research program, Hinterliter offers almost no evidence other than two one-sentence summaries that he alleges show that the modality principle cannot be replicated (i.e., Penney, 1975; Tabbers et al., 2004).
Even a cursory review of our research shows that our research team has been committed to replication. As shown in Table 1, we have tested each of our multimedia design principles multiple times. Certainly, we and others have not always obtained positive results, but these cases form the basis for identifying what we call boundary conditions--that is, situations in which a principle is more or less likely to apply. Identification of boundary conditions represents a sign of the growing maturity of research on multimedia learning, and offers a basis for clarifying, refining, and adding to the cognitive theory of multimedia learning.
Multimedia Learning Principles Adapted from Mayer (2021)
|1. Coherence Principle: People learn better when extraneous material is excluded rather than included.||0.86||18 of 19|
|2. Signaling Principle: People learn better when cues are added that highlight the organization of the essential material.||0.69||15 of 16|
|3. Redundancy Principle: People do not learn better when printed text is added to graphics and narration.||0.10||8 of 12|
|4. Spatial Contiguity Principle: People learn better when corresponding words and pictures are presented near rather than far from each other on the page or screen.||0.82||9 of 9|
|5. Temporal Contiguity Principle: People learn better when corresponding words and pictures are presented simultaneously rather than successively. ||1.31||8 of 8|
|6. Segmenting Principle: People learn better when a multimedia lesson is presented in user-paced segments rather than as a continuous unit.||0.67||7 of 7|
|7. Pretraining Principle: People learn better from a multimedia lesson when they know the names and characteristics of the main concepts.||0.78||10 of 10|
|8. Modality Principle. People learn better from graphics and narration than from the graphics and on-screen text.||1.00||18 of 19|
|9. Multimedia Principle: People learn better from words and pictures than from words alone. ||1.35||13 of 13|
|10. Personalized Principle. People learn better from multimedia lessons when words are in conversational style rather than formal style.||1.00||13 of 15|
|11. Voice Principle. People learn better when the narration in multimedia lessons is spoken in a friendly human voice rather than a machine voice.||0.74||6 of 7|
|12. Image Principle. People do not necessarily learn better from a multimedia lesson when the speaker's image is added to the screen. ||0.20||4 of 7|
|13. Embodiment Principle: People learn more deeply from multimedia presentations when an onscreen instructor displays high embodiment rather than low embodiment.||0.58||15 of 17|
|14. Immersion Principle: People do not necessarily learn better in 3D immersive virtual reality than with a corresponding 2D desktop presentation. ||-0.10||6 of 9|
|15. Generative Activity Principle. People learn better when they are guided in carrying out generative learning activities during learning.||0.71||37 of 44|
For example, the modality principle does not apply when there are words but no graphics, consistent with Penny's (1975) early work with word lists, or when the lesson is self-paced, consistent with Tabbers et al. (2004). As chronicled by Fiorella and Mayer (2022), across 76 studies testing the modality principle by researchers around the globe, most failures to find positive effects were consistent with predictions that could be derived from the cognitive theory of multimedia learning. For example, the modality principle is predicted to be diminished or eliminated when the demands on working memory are low--such as when the lesson is slow-paced or self-paced, the material is simple, or the learners have sufficient prior knowledge.
The detection of theory-grounded boundary conditions for the modality principle does not necessarily indicate a lack of empirical support for the cognitive theory of multimedia learning, but rather helps us gain better insights into how it should be best applied. The principles listed in Table 1 should not be considered as rigid rules to be applied in all situations, but rather should be adapted to situations in line with what we know about how people learn as reflected in the cognitive theory of multimedia learning. Overall, I agree that the modality principle has not been found in all studies, but when an effect is not found it is worthwhile to determine whether this can be explained by CTML. In addition, overall, across the 76 studies reviewed by Fiorella and Mayer (2022), the median effect size favoring the modality principle was d = 0.65, which is in the medium-to-large range. I conclude that Hinterliter's third criticism is unwarranted and misleading.
Criticism 4: The Approach is Based on the Wrong Ism
Hinderliter's fourth criticism is that our work is based on the wrong ideological approach. In particular, he calls us "cognitivists" and suggests the true ism should be "constructivism". For example, he misrepresents CTML as being based on the false idea that the human mind works like a computer: "Constructivists may consider the conceptualization of students' minds as analogous to computers to be a gross oversimplification, but CTML is among many theories advanced by cognitivist thinkers that remain embedded within modern instructional practice" (p. 64). As another example, he misrepresents CTML as being opposed to fitting instruction to the needs of individual learners: "Constructivist pedagogy recommends the customization of lesson plans in order to suit the unique individuality of each learner, while cognitivism seeks to determine the 'true' manner in which students learn new information en route to global prescriptions that will benefit all" (p. 64).
As I have repeated noted in my writings, I consider the cognitive theory of multimedia learning to have its roots in constructivist theories of learning, which assume that learning involves actively attending to relevant aspects of incoming information, mentally reorganizing presented material into a coherent representation, and integrating what is presented with what the learner already knows (Mayer, 2021, 2022). In short, CTML is based on the fundamental idea that we do not learn like computers, but rather meaningful learning is a constructive process of sense making. Similarly, our work is based on the idea that different learners come to the learning task with different kinds of prior knowledge and different information processing capabilities; therefore, it is useful to consider that different principles may apply for different types of learners. In recent years, as the field matures, we have been able to focus on what I call boundary conditions for when each multimedia design principle most strongly applies including specifying for whom a principle is most relevant. Thus, Hinderliter grossly misrepresents CTML by accusing it of being based on equating the human mind with a computer and on advocating a one-size fits all approach to instructional design.
Lastly, Criticism 4 is based on the idea that contributions to our field should be judged based on their ideological approach. I have long held that searching for the perfect ism is not a productive activity for educational psychology (Mayer, 1997). I do not think our field is served by taking an ideological approach rather than a scientific approach based on testing theories against research evidence. Overall, I conclude that Hinterliter's fourth claim is not only unwarranted but also misguided.
Criticism 5: The Research Only Deals with Typical Students
Hinderliter's final claim is that "the broad applicability of CTML's principles is often in conflict with issues of accessibility" particularly for learners with challenges such as "vision impairment, hearing impairment, cognitive impairments, or low language proficiency" (p. 64). He proposes focusing on "neurotypical native English speakers raises concerns regarding the generalizability of such findings" (p. 65).
I agree that most of our research has not targeted special populations, although we have examined the applicability of the redundancy principle and multimedia principle for students learning in their second language (Lee & Mayer, 2017; Mayer & Lee, 2015; Mayer, Lee, & Peebles, 2014). In addition, some of our collaborative studies also involve learning in languages other than English (Li et al., 2019, 2022; Wang et al., 2018; Xie et al., 2019). Certainly, an important next step in research on multimedia design principles is to examine how they apply to students who are not neurotypical, including students with specific learning challenges. Overall, I conclude that Criticism 5 is warranted, but rather than suggesting a flaw in CTML it reflects a path for future research that can strengthen it.
Looking to the Future of Multimedia Instructional Design
Don't get me wrong, I am not against criticism (although I prefer constructive criticism rather than unwarranted criticism). In this section, I sketch out my own constructive suggestions concerning areas for improvement in research on multimedia instructional design that warrant attention in the future.
- Broaden the context of learning: Conduct studies in authentic learning venues such as classrooms and online courses, and with new media such as virtual reality, interactive simulations and games, animated pedagogical agents, instructional video, and narrated animation.
- Broaden the learner population: Include students that represent the diversity of the student population in terms of age, gender, race, ethnicity, language proficiency, nationality, and individual differences in terms of cognitive, physical, and sensory skills.
- Broaden the measures of learning process and outcomes: In addition to transfer and retention posttests, incorporate embedded tests of learning outcomes, as well as biometric, cognitive neuroscience, eye-tracking, and data mining measures of learning processes.
- Incorporate affective and social processes: Expand CTML to include the learner's affective and social processes during learning.
- Incorporate motivational processes: Expand CTML to include the learner's motivational processes including interest, self-efficacy, and beliefs.
- Incorporate metacognitive processes: Expand CTML to include the learner's metacognitive processes, including learning strategies and executive function skills.
In the future, I expect to see an ever increasing empirical research base, a growing set of evidence-based design principles, more refined descriptions of boundary conditions, and a more inclusive theory of multimedia learning.
In spite of the many unwarranted criticisms I have confronted in this chapter, I prefer to frame Hinderliter's chapter as a well-meaning call for improvement in research and theory in our field. I appreciate Hinderliter's interest in our work and look forward to our field's continuing efforts to build principles of instructional design that are based on research evidence, grounded in theories of how people learning, and relevant to today's educational challenges.
Preparation of this paper was supported by grant N00014-21-1-2047 from the Office of Naval Research.
Fiorella, L. & Mayer, R. E. (2022). Principles for managing essential processing in multimedia learning. In R. E. Mayer & L. Fiorella (eds.), The Cambridge handbook of multimedia learning (3rd ed; pp. 243-260). Cambridge University Press.
Hinderliter, H. (2021). The case for rethinking multimedia. In B. Hokanson, M. Exter, A. Grincewicz, M. Schmidt, & A. A. Tawfik, A.A. (eds.), Learning: Design, engagement and definition (pp. 57-68). Springer.
Lee, H., & Mayer, R. E. (2018). Fostering learning from instructional video in a second language. Applied Cognitive Psychology, 32, 648-654. https://doi.org/10.1002/acp.3436
Li, W., Wang, F., Mayer, R.E., & Liu, H. (2019). Getting the point: Which kinds of gestures by pedagogical agents improve multimedia learning? Journal of Educational Psychology, 111 (8), 1382-1395. https://doi.org/10.1037/edu0000352
Li, W., Wang, F., Mayer, R. E., & Liu, T. (2022). Animated pedagogical agents enhance learning outcomes and brain activity during learning. Journal of Computer Assisted Learning, 38 (3), 621-637. https://doi.org/10.1111/jcal.12634
Mayer, R. E. (1997). Searching for the perfect ism: An unproductive activity for educational research. Issues in Education, 3 (2), 225-228.
Mayer, R. E. (2002). Multimedia learning. Psychology of Learning and Motivation, 41, 85-139. https://doi.org/10.1016/S0079-7421(02)80005-6
Mayer, R. E. (2005). Cognitive theory of multimedia learning. In R. E. Mayer (ed.) The Cambridge handbook of multimedia learning (pp. 202-212). Cambridge University Press.
Mayer, R. E. (2009). Constructivism as a theory of learning versus as a prescription for instruction. In S. Tobias & T. M. Duffy (Eds.). Constructivist theory applied to education: Success or failure? (pp. 184-200). Erlbaum.
Mayer, R. E. (2014). Multimedia learning. In J. Michael Spector, M. David Merrill, E. Jan, & M. J. Bishop (eds.), Handbook of research on educational communications and technology (4th ed., pp. 385-399). Springer.
Mayer, R E. (2021). Multimedia learning. Cambridge University Press.
Mayer, R. E. (2022). Cognitive theory of multimedia learning. In R. E. Mayer & L. Fiorella (eds.). The Cambridge handbook of multimedia learning (3rd ed; pp. 57-72). Cambridge University Press.
Mayer, R. E. & Fiorella, L. (2022). (eds.). The Cambridge handbook of multimedia learning I3rd ed). Cambridge University Press.
Mayer, R. E., & Lee, H. (2015). Visual aids to learning in a second language: Adding video to an audio lecture. Applied Cognitive Psychology, 29, 445-454. https://doi.org/10.1002/acp.3123
Mayer, R. E., Lee, H., & Peebles, A. (2014). Multimedia learning in a second language: A cognitive load perspective. Applied Cognitive Psychology, 28 (5), 653-660. https://doi.org/10.1002/acp.3050
Penney, C. G. (1975). Modality effects in short-term verbal memory. Psychological Bulletin, 82 (1), 68–84. https://psycnet.apa.org/doi/10.1037/h0076166
Tabbers, H. K., Martens, R. L., & van Merriënboer, J. J. G. (2004). Multimedia instructions and cognitive load theory: Effects of modality and cueing. British Journal of Educational Psychology, 74 (1), 71–81. https://doi.org/10.1348/000709904322848824
Wang, F., Li, W., Mayer, R. E., & Liu, H. (2018). Animated pedagogical agents as aids in multimedia learning: Effects on eye-fixations during learning and learning outcomes. Journal of Educational Psychology, 110 (2), 250-268. https://psycnet.apa.org/doi/10.1037/edu0000221
Xie, H., Wang, F., Mayer, R. E., & Zhou, Z. (2019). Coordinating visual and auditory cueing in multimedia learning. Journal of Educational Psychology, 111 (2), 235-255. https://psycnet.apa.org/doi/10.1037/edu0000285