Introduction

Researchers have conducted a myriad of studies focusing on the effects of virtual reality (VR) on student attitudes in classes. However, studies on VR’s effect on student performance are lacking (Johnson et al., 2019).  Of the few published studies which focus on VR’s effect on student performance, researchers note the use of VR is happening across all grade levels and subject areas within K-12 and higher education with mostly positive effects on students, though not all the time. Bower and Jong (2020) also suggest “given the expanding use of IVR in education, the wide variety of disciplinary, age, technological and pedagogical contexts in which it can operate…, there is undoubtedly pressing need for more IVR educational research." In other words, more research is needed to determine whether incorporating VR in a course affects student performance.

Purpose of the study

The purpose of this article is to determine whether student performance in a four-week, online, higher education Instructional Design class improves with the integration of VR.  In previous iterations of the IDT574: Digital Media and Learning Applications course, students engaged in “audio” learning activities for the first week. For Week 2, they participated in “video” learning activities. For Week 3, they learn about “interactive media” learning activities, which included a brief discussion about VR, but not the actual usage of it. For Week 4, students applied the concepts, principles and theories from the previous three weeks, integrating them all together in a final project.  The final project was a “video proposal” where students create a video pitch on how they could apply the modalities of audio, video, and interactive video to enhance patrons’ experience while visiting the Osceola County Welcome Center and History Museum.

Starting in May 2022, the researchers incorporated the use of VR into Week 3 of the course, where students interacted with VR applications, such as YouTube 360. The researchers specifically chose higher education students since they are currently working with Master students in the Instructional Design & Technology Master of Science degree program and are involved in their instruction. In addition to contributing to the general body of knowledge, the researchers wanted to explore various avenues of enhancing their online degree program, including the incorporation of VR into the third and fourth week of their online classes, and to determine whether incorporating VR affects their students’ performance.  The researchers specifically focused on the following research questions,  developed from the IDT574 course learning outcomes:

1. Does including VR in an online class help students consider designing and delivering instructional content using a variety of software applications?
2. Can incorporating VR in an online class encourage students to effectively present information and data visually and verbally?
3. Does using VR in an online class motivate students to better assess the effectiveness of media used in an instructional module?

Literature review

VR integration into training environments and courses provides numerous benefits to adult learners. These benefits include increased learner engagement, greater motivation, more support for differentiated learning styles, increased sense of belonging, and more collaboration among peers (NC State, 2024). In other words, the recognized benefits of VR integration are seen to positively affect the affective state of adult learners. Similarly, current research which examines the integration of VR into face-to-face or online educational environments confirms its positive effect on the affective state of adult students after they utilize VR in their learning (Johnson et al., 2019).  

However, a limited number of studies exist which look at student performance in relation to their use of VR in learning environments (Bower & Jong, 2020). Of the studies which investigate VR’s effect on student performance, studies indicate VR integration led to development in knowledge and skill as well as prolonged learning over a period of time in younger students (Wu et al., 2020).  In studies involving older students, results indicate positive effects on learning and motivation (Di Natale et al., 2020). Yet, another recent study indicates VR end users in higher education have a relatively low perception of its usefulness and UX (Matsika & Zhou, 2021). In this study, researchers from Full Sail University hope to add on to this body of knowledge. 

Methodology

The researchers decided to take an Action Research (AR) approach for this study (see Figure 1). The AR process includes the steps of “planning, acting, observing, and reflecting on the results generated from a particular project or body of work” (Dick, 1999; Zuber-Skerritt, 2002).  The reason researchers chose this methodology is that course development is still ongoing and is a work in progress. 

Figure 1

Timeline for the action research of the study

Data collection design

Methods included comparing past and current grades, examining course evaluation ratings, distributing open-ended surveys to the enrolled students, and coding student-instructor communication through one-on-one Zoom sessions (see Table 1). 

Table 1

Data sources

Participants

The population of this study consists of Instructional Design Master students enrolled in a Digital Media and Learning Applications class at Full Sail University.  A total of 56 students participated in the study.  Twenty-two of the students were from the IDT574 classes which did not incorporate VR into their third week assignment, and 34 of the students were from the IDT574 classes which incorporated VR into their third week assignment.

Findings

When comparing past and current grades of students before and after the implementation of VR within the IDT574 course, the researchers noticed a small increase in grades; however this increase was not significant enough to indicate studet performance improved due to use of VR. Similarly, the course satisfaction ratings within the course evaluations rose slightly, but the researchers did not see this as a significant improvement.  

However, themes appeared when the researchers used constant comparative method to analyze the qualitative data from the student interviews and open-ended surveys. After coding the data and analyzing it, the themes of prior experiences, learning styles, motivation, and inspiration appeared when students discussed the effects of VR on their performance in the class. A few students indicated prior experiences with VR caused them to already have a negative perception of VR and therefore resist using it within the class. As one student said, "Performance-wise I still feel [VR] isn't ideal, but I am being apprehensive due to my own personal opinion [and] experience with it." Others indicated similar feelings during their interviews. Students also mentioned their use of VR in assignments was based off their own personal learning styles. Some felt using VR matched their learning style and therefore helped their performance, while others felt it was not a match and did not assist them in their overall performance.  

Others stated that designing with VR for their assignments motivated and inspired them in the class, which aligns with results from other studies which feature the addition of VR in the classroom (Soto et al., 2020). A participant share "I would say that I... improved because I never thought about, um, the Virtual Reality part [until] now...Some of those modules [from my workplace] ...could be so much more engaging if they actually was having us looking in a virtual reality."  Another student stated, "[VR] gave me so many ideas on the things that I can do to, um, liven and engage...this week's project."  Interestingly, other themes appeared in the qualitative data to indicate different factors were responsible for increased student performance besides VR. These themes included peer interaction, real-world context, student examples, problem-solving situations, learner support, and next-step opportunities.  Suggestions from the student interviews were added over the eight months of the Action Research process to further develop the course.

Conclusion

Lack of significant increase occurred within the course satisfaction numbers and the grades themselves.  However, the increase in rigor of the course may have hindered the numbers and grades from considerably increasing. A further look into the qualitative data indicates that VR only affected the performance of those who have minimal prior experience with VR or those who believed their learning style matched the way VR presents information within a particular learning environment.  The researcher of this study believe more research and participants are needed to explore the connection among performance, VR, and learning styles coupled with prior experiences.

What was also of interest to the researchers was the other themes revealed in the one-on-one interviews not related to VR. The themes of peer interaction, real-world context, student examples, problem-solving situations, learner support, and next-step opportunities occurred at various times throughout the interviews while asking students questions about what affected their performance. Over eight months of the study, the course instructor implemented these suggestions. The researchers continued to interview the students about their performance while the instructor applied these course improvements. The researchers plan to analyze this additional data and publish their results in future studies.

References

Bower, M., & Jong, M. S.-Y. (2020). Immersive virtual reality in education. British Journal of Educational Technology, 51(6), 1981–1990. https://doi.org/10.1111/bjet.13038 

Di Natale, A. F., Repetto, C., Riva, G., & Villani, D. (2020). Immersive virtual reality in K‐12 and higher education: A 10‐year systematic review of empirical research. British Journal of Educational Technology, 51(6), 2006–2033. Academic Search Complete.

Dick, R. (1999). Rigour without numbers: The potential of dialectical processes as qualitative research tools (3rd ed.). Bloomington: Interchange.

Johnson, J., McDonald, R. H., Lischer-Katz, Z., Hall, N., Cook, M., Hardesty, J., & Carlisle, T. (2019). Challenges and strategies for educational virtual reality: Results of an expert-led forum on 3D/VR Technologies across academic institutions. Information Technology and Libraries (ITAL), 4(38), 25–48. https://doi.org/10.6017/ital.v38i4.11075

Matsika, C., & Zhou, M. (2021). Factors affecting the adoption and use of AVR technology in higher and tertiary education. Technology in Society, 67, 101694. https://doi.org/10.1016/j.techsoc.2021.101694 

NC State University. (2024). Current virtual reality best practices for the classroom. NC State University. Retrieved February 4, 2023, from https://teaching-resources.delta.ncsu.edu/current-virtual-reality-best-practices-for-the-classroom/ 

Reiser, R., & Dempsey, J. (2017). Trends and issues in instructional design and technology (4th edition). New York: Pearson.

Soto, N. C., Navas-Parejo, M. R., & Guerrero, A. M. (2020). Virtual reality and motivation in the educational context: Bibliometric study of the last twenty years from Scopus. Alteridad, 15(1), 47. https://doi.org/10.17163/alt.v15n1.2020.04 

Wu, B., Yu, X., & Gu, X. (2020). Effectiveness of immersive virtual reality using head‐mounted displays on learning performance: A meta‐analysis. British Journal of Educational Technology, 51(6). https://doi.org/10.1111/bjet.13023

Zuber-Skerritt, O. (2002). A model for designing action learning and action research programs. The Learning Organization, 9(4), 143–149. https://doi.org/10.1108/09696470210428868