PROMPTING LANGUAGE MATTERS: A STUDY OF CHATGPT’S GRAMMAR FEEDBACK IN VIETNAMESE AND ENGLISH
Main Article Content
Abstract
This research investigates whether ChatGPT tends to give explanations that focus more on exam tips (Rule-Centric - RC) or communicative functions (Meaning-Enriched - ME) when it is asked to give explanations to grammar exercises in Vietnamese and in English. The researcher developed four sets of grammar-focused multiple-choice questions on tenses, each of which consisted of 10 items. Each question was then duplicated into two copies: one with the instruction in Vietnamese and one in English, with the rest identical. The questions were sent to ChatGPT five times, which produced a total of 200 responses per language. Two independent raters then rated the explanation for each question as either RC or ME. The analysis showed a strong tendency for the Vietnamese prompts to produce more rule-centric explanations, while English prompts often resulted in more meaning-enriched results, except for the Present Simple set with the Vietnamese prompts, where the percentage of ME responses was considerably higher than for the others. Overall, these findings suggest that the prompt language may influence ChatGPT's grammar explanations, and this raises concerns about the reinforcement of exam-oriented thinking in students and teachers. The study also discusses implications for teachers, parents, and learners on the use of ChatGPT and other Large Language Models (LLMs) in exam-driven contexts like in Vietnam.
Keywords
ChatGPT, large language models (LLMs), EFL in Vietnam, washback, exam preparation
Article Details
References
Alshehri, E. (2017). Using learners’ first language in EFL classrooms. IAFOR Journal of Language Learning, 3(1), 20–36. https://files.eric.ed.gov/fulltext/EJ1167274.pdf
Bailey, K. M. (1996). Working for washback: A review of the washback concept in language testing. Language Testing, 13(3), 257–279. https://doi.org/10.1177/026553229601300303
Brown, J. D. (2005). Testing in language programs. McGraw-Hill Companies.
Ellis, R. (2006). Current issues in the teaching of grammar: An SLA perspective. TESOL Quarterly, 40(1), 83–107. https://doi.org/10.2307/40264512
Ellis, R. (2016). Focus on form: A critical review. Language Teaching, 49(3), 341–374. https://doi.org/10.1177/1362168816628627
Kasneci, E., Sessler, K., Küchemenn, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., Krusche, S., Kutyniok, G., Michaeli, T., Nerdel, C., Pfeffer, J., Poquet, O., Sailer, M., Schmidt, A., Seidel, T., Stadler, M., & Kasneci, G. (2023). ChatGPT for good? On opportunities and challenges of large language models for education. Learning and Individual Differences, 103, 102274. https://doi.org/10.1016/j.lindif.2023.102274
Larsen-Freeman, D. (2001). Teaching grammar. In M. Celce-Murcia (Ed.), Teaching English as a second or foreign language (3rd ed., pp. 251–266). Heinle & Heinle.
Nguyen, M. H. T. (2020). Beyond the washback of the English National Exam on learning: Subsequent impact on language learning strategies of first-year English majors. Language Education & Assessment, 3(2), 38–57. https://doi.org/10.29140/lea.v3n2.356
Norris, J. M., & Ortega, L. (2000). Effectiveness of L2 instruction: A research synthesis and quantitative meta-analysis. Language Learning, 50(3), 417–528. https://doi.org/10.1111/0023-8333.00136
O’Brien, M. J., Alsmadi, I., & Bentley, R. A., Tuba, M. (2024). OpenAI ChatGPT and biased information in higher education. In M. Labby (Ed.), Empowering digital education with ChatGPT (pp. 223–234). Chapman and Hall/CRC.
Shaffer, C. (1989). A comparison of inductive and deductive approaches to teaching foreign languages. Modern Language Journal, 73(4), 395–403. https://doi.org/10.2307/326874
Solaiman, I., & Dennison, C. (2021). Process for adapting language models to society (PALMS) with values-targeted datasets. In Proceedings of the 35th International Conference on Neural Information Processing Systems (pp. 5861–5873). https://proceedings.neurips.cc/paper/2021/file/2e855f9489df0712b4bd8ea9e2848c5a-Paper.pdf
Widodo, H. P. (2006). Approaches and procedures for teaching grammar. English Teaching: Practice and Critique, 5(1), 122–141. https://edlinked.soe.waikato.ac.nz/journal/files/etpc/files/2006v5n1nar1.pdf