Traditionally, language teaching and learning occur in a classroom setting, where an educator would facilitate and deliver a lesson, and getting the students to practice speaking with one another. A paradigm shift would come with the advent of the Internet and search engines would provide greater accessibility and resources to prospective language learners, transforming the ecosystem of language learning methods. However, the 2020s would bring yet another paradigm shift, with the introduction of large language models (LLMs) and generative artificial intelligence (GenAI) into multiple facets of digital life, from chatbots to so-called ‘vibe-coding’, to growing hype and disdain.

The shift towards generated lessons and exercises in popular language learning applications such as Duolingo has prompted (ha, see what I did there?) me to examine the general themes and attitudes surrounding the use of GenAI and LLMs in the processes of language teaching and language acquisition.

To get an overall picture, we will take a look into a scoping review on this very topic. A scoping literature review maps the potential size, scope, and trends of available research literature in a given topic. Given the relatively nascent nature of such a learning and teaching method, it is perhaps important to know what researchers are beginning to deal with when proposing, designing, and conducting their respective studies. This scoping review is written by L. Law, and is published in 2024 in the journal Computers and Education Open. As a disclaimer, Law’s scoping review disclosed the use GenAI to improve the readability and language used in the paper, but the author claimed to have reviewed and edited the content as needed. The use of GenAI programmes in the planning and conduct of the scoping review was not disclosed.

Law wanted to address some questions through a scoping review, which were the key terms, popular research fields, types of languages and education used in research, the attitudes towards GenAI in language learning, and the potential challenges and benefits of this method in language teaching and learning. To do this, Law narrowed the time period of interest to the introduction of the Transformer, until 2023, the time of writing. GenAI and LLMs have exploded in interest in 2022, but they are the product of years of underlying research and development. The Transformer is a type of neural network architecture, which first modern version was detailed in a publication in 2017. This formed the start of the time period of studies to be considered in Law’s scoping review.

Literature searches were made in four literature databases, as well as grey literature databases, which are papers which are not published by academic journals. Grey literature may include technical reports, working papers, and government documents, and given the nascent nature of this research topic, grey literature could potentially contribute some perspectives beyond journal publications, provided they meet pre-defined inclusion criteria and validity for the scoping review process. The search process yielded a total of 224 hits, amongst which 195 were unique. However, only 41 studies were ultimately included in the scoping review, with the rest rejected for failing to meet pre-defined criteria (such as English papers, papers focusing on GenAI or language education, accessibility, and quality).

Nearly all of these included papers were published in 2023, with one published in 2022. Additionally, a considerable proportion of these papers were from East Asia, though there were several papers with no specific or identified data origins. A majority of these papers were empirical studies, with a roughly equivalent mix of quantitative, qualitative, and mixed methods studies, something that might warrant a closer look into when it comes to future Journal Club essays here. Many of these studies also center around the study of English as a foreign language at different stages of education, and understandably so, since English is perhaps amongst the most widely studied foreign languages globally. General teaching and learning seems to be the main area of investigation, with particular focus placed on the written medium. Other foci include teaching and learning policy and ethics of GenAI use in language teaching and learning.

A majority of these studies demonstrated a positive attitude to the use of GenAI in language teaching and learning, with only one included study demonstrating a negative attitude. Even studies with balanced or mixed attitudes are a rarity too. We can chalk it up to the novelty and the initial openness to adopting such technologies in these applications, but it would be worth to understand the arguments for these positive attitudes. These tend to revolve around topics related to productivity and some psychological aspects. By feeding models data regarding language proficiency through different means, such as written texts, GenAI and LLMs would be able to tailor and adapt to the user’s current proficiency level, and give a more personalised language learning experience. Some studies suggested that this might even ‘surpass the capabilities of human class teachers’, but I am quite sceptical of the overall effectiveness in these educational settings. As these studies focus more on the written medium, we do not hear much about arguments regarding speaking anxiety, a challenge faced by foreign language learners when speaking and listening in their target languages.

On the teaching side, GenAI has been purported to aid the grading and evaluation process in student assignments. While it does help teachers by reducing the workload otherwise spent on grading student assignments, I am not bought by the arguments that GenAI offers meaningful or productive feedback, since these models are not really capable of assessing more advanced cognitive skills, nor are they actually capable of critical thinking. Essays, one of the most common student assignments, assess the students’ abilities to form arguments, and present them in a cohesive manner. Evaluation rubrics tend to focus on the critical thinking presented in the essay, such as the strength of arguments and rebuttals, which are normally out of scope of what GenAI can evaluate reliably. Law argued that GenAI programmes could target the lexical and syntactical parts of the essay, but in my opinion, these tend to occupy a lesser focus when evaluating essays. As such, as convenient a tool GenAI may market itself to be to educators, the arguments behind the capability of GenAI to evaluate student assignments, especially essays, appear weak.

This is not to undermine the grave dangers of the integration of GenAI programmes in the language learning and teaching processes, as the literature included in this review has suggested that the potential benefits purported by the studies might not actually outweigh the challenges and issues that GenAI poses. These arguments tend to revolve around academic dishonesty and motivation, as well as topics related to higher cognitive skills and critical thinking. For example, studies criticise the integration of GenAI in pedagogical processes for negatively affecting learner motivations, as it is substantially less effortful to simply generate an essay compared to writing one by hand. As such, one would be motivated by taking shortcuts than spending the time and effort to actually learn. But all these for an increase in productivity and reduction of stress, right? I find it weird that these articles tend to produce positive attitudes than those of caution when these concerns are laid out in full view, but this is perhaps due to the emphasis placed on productivity than other psychological aspects involved in the learning and teaching process.

Perhaps one of the reasons why there is such a positive attitude to this is the fact that most included studies pertained to English as a foreign language. There is an enormous amount of English-language data for LLMs to train on, through legitimate means and illegitimate or illegal means alike (not like GenAI companies care, of course). As such, for English language education, GenAI would have been more fine tuned to cater to this very aspect. However, for languages that do not have as large a training corpus as English, such as Fijian for example, I think that the attitudes to the integration of GenAI in teaching and learning Fijian would be much less positive. As English is amongst the most widely learned foreign languages in the world, perhaps it is not surprising that there would be positive attitudes to GenAI integration in this case.

This scoping review is the first of its kind in this field of research, and as the results suggest, it is a pretty nascent topic in academia. Nevertheless, Law’s scoping review has provided some insight on the existing research foci on GenAI use in language learning and teaching, highlighting the need for more empirical studies to better understand the effects of these tools and applications. As of yet, the few empirical studies which have been conducted focus more on the relatively short-term due to the novelty of this technology in these applications. Over time, we could encounter more empirical studies focusing on the long-term effects, intervention studies, and more systematic reviews targeting this research topic.

Ethical concerns do remain a sticking point, even mentioned in the literature highlighted by the scoping review. Most concerns target the direct implications of GenAI use in pedagogical applications, such as bias and plagiarism, but tend to miss out on the indirect implications such as data privacy. Should these intervention studies be conducted, how might the researchers ensure the security and privacy of the participants’ data when they use GenAI programmes for the studies? These gaps in ethical discussions could fall under a wider umbrella problem of the lack of AI literacy, as there could be a lack of understanding of the acquisition and use of data when GenAI is programmes are used. For educational professionals, Law suggested that they should have a certain baseline standard of AI literacy, although the criteria for this baseline was not defined or elaborated upon.

Overall, if you want to get an overview of the current, or more rather, the early days of attitudes and topics in the pedagogical applications of GenAI focusing on language learning and teaching, this scoping review is a must-read. It also provides further research avenues to build upon, by either contributing more empirical evidence in various pedagogical outcomes (think critical thinking, quality of writing, language proficiency, speaking, and the like), and over time, acute and chronic effects of such applications in these outcomes (perhaps language retention as well).

The Journal Article

Law, L. (2024) Application of generative artificial intelligence (GenAI) in language teaching and learning: A scoping literature review, Computers and Education Open, 6, pp.100174. https://doi.org/10.1016/j.caeo.2024.100174.