Introduction 

The pandemic compelled children nationwide to attend school remotely or engage in self-paced online learning using tablet computers, such as iPads and Chromebooks. Concerns naturally arose among educators when students used their own device for learning (Seemiller, 2017; Laxman & Holt, 2017) because the Internet provides ‘digital distractions,’ leading to student cyberloafing. This involves checking real-time updates, accessing irrelevant web content and applications, and playing games unrelated to course materials during their study (Dursun et al., 2018; Greenfield, 2017; Westervelt, 2016). Cyberloafing behaviors were more prevalent among upper-class students with greater internet seniority and those with personal computers (Arabaci, 2017), negatively affecting students’ performance (Dursun et al., 2018). Therefore, measures are needed to minimize digital distractions, consequently, cyberloafing to create a desirable learning environment for our school-aged children.

This study is mainly intended to examine the impact of digital distraction on student performance across courses, as well as the moderating effect of average student age within each course on the relationship between digital distraction and student performance, using Hierarchical Linear Modeling (HLM). The research questions are as follows:

1. Is there a mean difference in student performance within and between courses based on gender, the presence of digital distraction, and the total number of study days? 
2. Does the course-level factor (the average student age within each course) explain the differences in student performance between courses?

Method 

Learning context

ABC (pseudonym), a Korean company, offers an English language program for learners of all levels, from K-12 students to adults. This program allows learners to study at their own pace using a digital device and a course textbook. ABC’s program features a curriculum with multiple levels. Generally, older students tend to enroll in higher-level courses. While the company permits members to study its program on any mobile device, it also offers a restricted device. This device is a customized version of a commercial device with the same functions as other mobile devices but restricts the download of applications, access to web content, and use of the camera, phone, and messenger. This device only allows access to the company’s language program and a few pre-set applications like a dictionary. ABC maintains standardized course structures and test formats across all courses. 

Dataset

The researchers were given access to one month of de-identified learning data from all the company’s student members. The original dataset was refined to include only cases with more than 20 study days and courses with at least 10 cases. The final dataset consists of 11,237 cases from 95 courses. The subjects of this study were K-12 students aged between 6 and 19 in Korean age, which is equivalent to 5 to 18 in international age. A majority of students opted to use the company’s specially designed digital device, which aimed to limit digital distractions. The remaining 629 (5.6%) students used their own devices. Among the students, 5,448 (48.5%) were female, while 5,789 (51.55%) were male. 

Variables

The dependent variable was student performance, a continuous variable with a total score of 50. It was comprised of three different test scores: vocabulary (10), pronunciation (10), and 4-skills (30). Upon completing each course, students underwent vocabulary, pronunciation, and ‘4-skills’ test. The ‘4-skills’ test evaluated achievement in four English language domains. At level 1, three independent variables were used: gender, digital distraction, and study days. Gender (Male = 0, Female = 1) and digital distraction (no digital distraction = 0, digital distraction =1) were categorical variables. “No digital distraction” indicated students who used ABC’s device, which limited digital distractions. Study days were continuous variables ranging from 20 to 31. At level 2, the average student age within each course was employed. There was no missing data.

Data Analysis

HLM was employed for data analysis due to the hierarchical nature of the dataset, with students nested within courses. Using HLM 7.0, we initially assessed data suitability for multilevel analysis through a fully unconditional model, calculating the intraclass correlation coefficient (ICC) at about 4% (ICC = 4.65/ [4.65+112.30] = .04). This indicates a small proportion of performance variance linked to course-related factors. To account for both within and between variances, level-1 predictors (gender, digital distraction, study days) and a level-2 predictor (average student age within each course) were included. Descriptive statistics analysis utilized the ‘psych’ package in R Studio. 

Results

Descriptive Statistics

At level 1, the mean for study days was 23.16 with an SD of 2.671, and the mean for students’ performance was 32.94 out of 50 with an SD of 10.802. The skewness was within ±2, and kurtosis was within ±7, except for digital distraction. This indicates that the variables except digital distraction are normally distributed (See Table 1). 

Table 1 

Descriptive Statistics of the Level 1 Variables and the Level 2 Variable

Multilevel Modeling

All level-1 predictors were statistically significant (p<.001) (See Table 2). However, only digital distraction and study days had both fixed and random effects, while gender had only fixed effects. This indicates that female students generally performed better than male students, and this gender difference was consistent across courses. Therefore, in the final level-1 random coefficient model, we excluded gender random variance (u1j). 

Level-1 Analysis  

In the level-1 final model (Table 2), the results supported the research hypothesis under research question 1. First, there was a significant difference in student performance between groups based on the presence of digital distraction across courses after controlling for gender and study days. There was a negative relationship (=-1.81, p<.001). Since a group of students who used a digital distraction-free tablet was coded as 0, these students performed better than those who used their own device. Also, there was a positive relationship between study days and student performance across all courses, after controlling for gender and digital distraction (β=.56, p<.001). The proportion of level-1 variance was explained by 3% of the three level-1 predictors. 

Table 2 

Multilevel Model Summaries with Level-1 Predictors

Note. Parameter estimate standard errors or standard deviation listed in parentheses *p<.05, **p<.001 

Level-2 Analysis  

Next, to explain between-course variation, we introduced the average student age within each course as a level-2 predictor. This level 2 predictor was added to two level 1 predictors – digital distraction and study days. There was a moderating effect of the average student age on study days across courses (β=.01,  p<.05). However, there was no moderating effect of average student age on the effect of digital distraction (β=.30, p>.05). Thus, we can conclude that there was no significant between-course effect on the relationship between digital distraction and student performance based on the average student age within each course. The between course effect based on study days was also minimal because was only .01 (See Table 3). 

Table 3 

Multilevel Model Summaries with a Level-2 Predictor

Note. Parameter estimate standard errors or standard deviation listed in parentheses *p<.05, **p<.001 

Discussion and Implication

In this study, we examined the impact of digital distraction and possible cyberloafing on student performance within courses, also exploring the moderating effect of the average student age within each course on this relationship. At the student level, we considered gender, the presence of digital distraction on their chosen device, and the total number of study days as predictors of student performance. We found that all three variables were significant predictors of student performance on average. In general, female students performed better than male students, and those who used a digital distraction-free device outperformed those who used their own device after controlling for all the other predictors. Moreover, students who studied more days achieved better performance. 

Recognizing that students’ self-control or self-regulation skills may vary with their age, we conducted a multilevel analysis using the average student age within each course as a second-level predictor to explain variations between courses. However, student age did not moderate the relationship between digital distraction and student performance. It had only a slight moderating effect on the relationship between the total number of study days and student performance. This implies that students tend to perform better when there is less digital distraction and invest more time in their learning, regardless of their age, across all courses. 

This study is not without limitations, and these limitations may affect our final results at both level 1 and level 2. One of the limitations is that the majority of students chose to use the digital distraction-free device. Another limitation is the significant variation in the number of data points between courses. However, the study has positive implications for K-12 educators, emphasizing the importance of reducing digital distraction to improve student performance when using tablet computers for their studies. We recommend conducting a larger-scale study with more balanced data and consider additional group-level factors, such as socioeconomic status and the presence of literacy education, to provide further insights into the relationship between digital distraction and learner performance.  

References

Arabaci, I. B. (2017). Investigation Faculty of Education Students' Cyberloafing Behaviors in Terms of Various Variables. Turkish Online Journal of Educational Technology-TOJET, 16(1), 72-82. https://eric.ed.gov/?id=EJ1124916 

Dursun, O. O., Donmez, O., & Akbulut, Y. (2018). Predictors of cyberloafing among preservice information technology teachers. Contemporary Educational Technology, 9(1), 22–41. https://eric.ed.gov/?id=EJ1166432 

Greenfield, D. (2017, October 20). Digital Distraction: Internet and Smartphone Addiction. https://www.psychologytoday.com/blog/virtual-addictions/201710/digital-distraction-internet-and-smartphone-addiction 

Laxman, K., & Holt, C. (2017). Bring Your Own Device or Bring Your Own Distraction. International Journal on E-Learning, 16(3), 245-263.

Seemiller, C. (2017). Curbing Digital Distractions in the Classroom. Contemporary Educational Technology, 8(3), 214-231.

Westervelt, E. (2016, November 05). Learning In The Age of Digital Distraction. Retrieved February 08, 2018, from https://www.npr.org/sections/ed/2016/11/05/498477634/learning-in-the-age-of-digital-distraction