Background

Implementing chatbots in classrooms for learning English as a foreign language (EFL) has been advocated due to its known advantages for all-level students (Xu & Warschauer, 2020; Yang et al., 2022). It has been reported that interacting with chatbots is beneficial in terms of lowering anxiety, having more speaking opportunities, and thus developing communication skills (Chen et al., 2020; Hwang et al., 2022). Many studies, however, have identified some limitations of chatbots in terms of ultimately cultivating language proficiency (Dizon, 2020; Tai & Chen, 2022). In addition, some aspects of language learning with chatbots are still controversial. For example, there are still differences of opinion about how well chatbots can understand what learners say (Chen et al., 2020) and how designing multimodal chatbots affects the cognitive load of learners (Sweller, 2020). 

Multiple studies have pointed out the same flaws and problems with using chatbots, so it's time to take a closer look and figure out which parts of chatbots need to be improved for effective language learning. In this paper, we define chatbots as intellectual conversational agents with whom learners interact for language learning purposes, including Google Assistant and various other independently developed systems. This paper will provide directions for future researchers and teachers who intend to design better language learning experiences by focusing on relatively neglected aspects of chatbots in EFL classrooms.

Methods

We conducted a scoping review, analyzing existing literature on the topic, and extracting and charting information (Peters et al., 2022). To begin with, we searched relevant studies in databases including Web of Science, APA PsycINFO, Education Source, and ERIC, using search terms: ‘EFL’, ‘English learning’, ‘chatbot’, and ‘intelligent personal assistant’. We included studies that (1) were published after 2010, (2) adopted an empirical approach,  and (3) were written in English. A full-text review of 62 studies was done, and a summary was made using directed content analysis (Hsieh & Shannon, 2005) with the five conceptual foundations for the technology-enhanced learning environment as a guide. We synthesized the issues and limitations suggested in previous studies and provided potential breakthroughs for them, aggregating the ideas partially mentioned in studies scattered in the field. The original framework (Driscoll & Burner, 2021, p. 280) was adapted to reflect the interconnectivity of all foundations (Stenbom et al., 2016).

Figure 1

Five Conceptual Foundations for the Technology-enhanced Learning Environment

Discussion and Significance 

In the presented framework, the psychological foundation is concerned with learning theories about how learners interact with technology to acquire and organize knowledge. The novelty effect and learners losing interest in chatbots over time have been constantly reported (Huang et al., 2022). This can also affect learners’ perceptions of chatbots’ human likeness. Girouard-Hallam et al. (2021) observed that learners tend to anthropomorphize chatbots as they gain more knowledge about them. This is undesirable in terms of motivation, as Jeon (2021) noted in some cases, students held a negative attitude toward learning English when they perceived chatbots as machines. If students don’t feel like talking to a human partner, they tend to lose the willingness to communicate. Also, the jokes of chatbots lose appeal over time (Lopatovska, 2020), which calls for a need to build more human-like chatbots. Cultural foundations of the EFL environment play a role here, and Warschauer (2020) reported that ESL preschoolers in the United States perceived chatbots as a mixture between humans and machines. However, Lee and Jeon (2022), and Liu et al. (2022) said that EFL elementary students were more likely to give chatbots human traits. This can be due to the cultural differences between the ESL and EFL environments. Since the EFL environment does not allow much natural use of the target language, students tend to have high levels of anxiety, frustration, and aspiration toward communicating in English, making them more engaged in interacting with chatbots. However, this is not always the case, especially for low-proficiency learners. In terms of pedagogical foundation, many studies repeatedly reported the obstacles low-proficiency learners face when interacting with chatbots: not being able to take more initiative in dialogue (Wu et al., 2020) and adopt different communicative strategies (Chen et al., 2020), but abandon the conversation (Tai & Chen, 2022). It gets even worse when combined with issues from the technological foundations because when chatbots have difficulty understanding learners’ pronunciations, low-proficiency learners usually repeat exactly the same utterance or give up (Chen et Al., 2020). Lastly, although the penetration rate of mobile devices in K-12 schools has largely increased after the pandemic (Goteka, 2022), pragmatic foundations, such as space and time, are still some of the remaining issues to tackle to support language learning through interacting with chatbots. 

Four potential solutions emerged from the content analysis. First, chatbots need to provide translations for low-proficiency students. A chatbot being able to communicate in both L1 and L2 is time-saving (Tai, 2022) and decreases students’ cognitive load (Divekar et al., 2018). It also complies with the communicative language teaching approach (Richards, 2005), which allows flexible use of the native language to improve understanding of the target language. Second, using predefined commands for communication can be an adaptive strategy for low-proficiency learners. It helps learners quickly familiarize how chatbots work (Pham et al., 2018) and makes the interface less confusing, which often causes frustration (Moussalli & Cardoso, 2021). Third, designing multimodal feedback is beneficial for low-proficiency learners. It facilitates learners’ connection between aural and written forms of utterances (Lan & Liao, 2018) and lowers the level of anxiety when miscommunication happens (Tai & Chen, 2022). Lastly, various assistive strategies are currently explored in collaboration with chatbots, such as mind maps (Lin & Mubarok, 2021), supportive task design to assure learners’ understanding of chatbots (Dizon, 2020), and integration with social media (Belda-Medina & Calvo-Ferrer, 2022). 

Revisiting existing literature and suggesting directions for future studies is especially worthwhile at this point when AECT marks its 100th anniversary this year. In this study, we reflected on previous contributions that have established our fairly positive perspectives on using chatbots in language learning. This study will help future studies incorporate chatbots not repeating the same mistake and designing more holistic language learning experiences.

References

Adolphs, S. (2006). Introducing electronic text analysis: A practical guide for language and literary studies. Routledge.

Belda-Medina, J., & Calvo-Ferrer, J. R. (2022). Using Chatbots as AI Conversational Partners in Language Learning. Applied Sciences, 12(17), 8427. https://doi.org/10.3390/app12178427 

Chen, H. H. J., Yang, C. T. Y., & Lai, K. K. W. (2020). Investigating college EFL learners’ perceptions toward the use of Google Assistant for foreign language learning. Interactive Learning Environments, 31(3), 1335-1350. https://doi.org/10.1080/10494820.2020.1833043 

Divekar, R. R., Drozdal, J., Zhou, Y., Song, Z., Allen, D., Rouhani, R., Zhao, R., Zheng, S., Balagyozyan, L., & Su, H. (2018 Interaction challenges in AI equipped environments built to teach foreign languages through dialogue and task-completion [Paper presentation]. DIS 2018 - Proceedings of the 2018 Designing Interactive Systems Conference (pp. 597–610). https://doi.org/10.1145/3196709.3196717  

Dizon, G. (2020). Evaluating intelligent personal assistants for L2 listening and speaking development. Language Learning & Technology, 24(1), 16-26. https://doi.org/10125/44705 

Girouard‐Hallam, L. N., Streble, H. M., & Danovitch, J. H. (2021). Children's mental, social, and moral attributions toward a familiar digital voice assistant. Human Behavior and Emerging Technologies, 3(5), 1118-1131. https://doi.org/10.1002/hbe2.321

Goteka, P. (2022, June 23). Increased prevalence of mobile devices in K-12 schools. Mobileguardian.  Retrieved from https://www.mobileguardian.com/increased-prevalence-of-mobile-devices-in-k-12-schools/ 

Hsieh, H. F., & Shannon, S. E. (2005). Three approaches to qualitative content analysis. Qualitative health research, 15(9), 1277-1288. https://doi.org/10.1177/1049732305276687 

Huang, W., Hew, K. F., & Fryer, L. K. (2022). Chatbots for language learning—Are they really useful? A systematic review of chatbot‐supported language learning. Journal of Computer Assisted Learning, 38(1), 237-257. https://doi.org/10.1111/jcal.12610 

Hwang, W. Y., Guo, B. C., Hoang, A., Chang, C. C., & Wu, N. T. (2022). Facilitating authentic contextual EFL speaking and conversation with smart mechanisms and investigating its influence on learning achievements. Computer Assisted Language Learning, 1-27. https://doi.org/10.1080/09588221.2022.2095406 

Jeon, J. (2021). Exploring AI chatbot affordances in the EFL classroom: Young learners’ experiences and perspectives. Computer Assisted Language Learning, 1-26. https://doi.org/10.1080/09588221.2021.2021241 

Kastner, M., Tricco, A. C., Soobiah, C., Lillie, E., Perrier, L., Horsley, T., Welch, V., Cogo, E., Antony, J., & Straus, S. E. (2012). What is the most appropriate knowledge synthesis method to conduct a review? Protocol for a scoping review. BMC Medical Research Methodology, 12(1), 1-10. https://doi.org/10.1186/1471-2288-12-114 

Lan, Y. J., & Liao, C. Y. (2018). The effects of 3D immersion on CSL students’ listening comprehension. Innovation in Language Learning and Teaching, 12(1), 35–46. https://doi.org/10.1080/17501229.2018.1418242 

Lee, S., & Jeon, J. (2022). Visualizing a disembodied agent: Young EFL learners’ perceptions of voice-controlled conversational agents as language partners. Computer Assisted Language Learning, 1-26. https://doi.org/10.1080/09588221.2022.2067182 

Lin, C. J., & Mubarok, H. (2021). Learning analytics for investigating the mind map-guided AI chatbot approach in an EFL flipped speaking classroom. Educational Technology & Society, 24(4), 16-35. https://www.jstor.org/stable/48629242 

Liu, C. C., Liao, M. G., Chang, C. H., & Lin, H. M. (2022). An analysis of children's interaction with an AI chatbot and its impact on their interest in reading. Computers & Education, 189, 104576. https://doi.org/10.1016/j.compedu.2022.104576 

Lopatovska, I. (2020, March). Personality dimensions of intelligent personal assistants. In Proceedings of the 2020 Conference on Human Information Interaction and Retrieval (pp. 333-337). https://doi.org/10.1145/3343413.3377993 

Moussalli, S., & Cardoso, W. (2021). Intelligent personal assistants and L2 pronunciation development:
focus on English past -ed. In N. Zoghlami, C. Brudermann, C. Sarré, M. Grosbois, L. Bradley, & S. Thouësny (Eds), CALL and professionalisation: short papers from EUROCALL 2021 (pp. 226-231). Research-publishing.net. https://doi.org/10.14705/rpnet.2021.54.1337 

Peters, M. D., Godfrey, C., McInerney, P., Khalil, H., Larsen, P., Marnie, C., Pollock, D., Tricco, A. C., & Munn, Z. (2022). Best practice guidance and reporting items for the development of scoping review protocols. JBI evidence synthesis, 20(4), 953-968. https://doi.org/10.11124/jbies-21-00242 

Pham, X. L., Pham, T., Nguyen, Q. M., Nguyen, T. H., & Cao, T. T. H. (2018, November). Chatbot as an intelligent personal assistant for mobile language learning. In Proceedings of the 2018 2nd International Conference on Education and E-Learning (pp. 16-21). https://doi.org/10.1145/3291078.3291115 

Richards, J. C. (2005). Communicative language teaching today. SEAMEO Regional Language Centre.

Sweller, J. (2020). Cognitive load theory and educational technology. Educational Technology Research and Development, 68(1), 1-16. https://doi.org/10.1007/s11423-019-09701-3 

Tai, T. Y. (2022). Effects of intelligent personal assistants on EFL learners’ oral proficiency outside the classroom. Computer Assisted Language Learning, 1-30. https://doi.org/10.1080/09588221.2022.2075013 

Tai, T. Y., & Chen, H. H. J. (2022). The impact of intelligent personal assistants on adolescent EFL learners’ listening comprehension. Computer Assisted Language Learning, 1-28. https://doi.org/10.1080/09588221.2022.2040536 

Wu, Y., Rough, D., Bleakley, A., Edwards, J., Cooney, O., Doyle, P. R., Clark, L., & Cowan, B. R. (2020, October). See what I’m saying? Comparing intelligent personal assistant use for native and non-native language speakers. In MobileHCI ‘20: 22nd International Conference on Human-Computer Interaction with Mobile Devices and Services (pp. 1–9). ACM. https://doi.org/10.1145/3379503.3403563 

Xu, Y., & Warschauer, M. (2020, April). What are you talking to?: Understanding children's perceptions of conversational agents. In Proceedings of the 2020 CHI conference on human factors in computing systems (pp. 1-13). https://doi.org/10.1145/3313831.3376416 

Yang, H., Kim, H., Lee, J. H., & Shin, D. (2022). Implementation of an AI chatbot as an English conversation partner in EFL speaking classes. ReCALL, 34(3), 327-343. https://doi.org/10.1017/S0958344022000039