EXAMINERS’ PERCEPTIONS OF A COMPUTER-ASSISTED ENGLISH LANGUAGE TEST
Main Article Content
Abstract
Computer-assisted language testing (CALT) is of great development in the last ten years thanks to the technological advances in the 4.0 era and due to the COVID-19. Increasingly, test takers - including young learners - are becoming accustomed to completing both standardized and classroom-based assessments online or with digital support. Widely recognized platforms such as IELTS, APTIS, and Duolingo exemplify this shift toward digital testing environments. Simultaneously, educators are gaining experience in transitioning from traditional paper-based assessments to computer-based formats. This evolution raises important questions about how examiners perceive and assess language performance in digital contexts. It is noteworthy to listen to examiners’ voices in regard to evaluating performances on computers or with computer support (Forrester, 2020; Yulianto & Mujtahid, 2021). This study investigates Vietnamese examiners’ perceptions on a specific computer-based English proficiency test. The qualitative data collected from focus-group discussions with six examiners provided insightful information on their perceptions and experiences on the test. Their insights offer a nuanced understanding of how they experienced digital interfaces, and how test takers are assessed with such an assessment mode. The findings from this research will contribute to a deeper understanding of the human dimension in CALT, highlighting both opportunities and challenges in digital assessment, especially the role of examiner agency and reflective practice in shaping reliable assessment.
Keywords
computer-assisted language testing, examiners’ perceptions, speaking test
Article Details
References
Bachman, L. F., & Palmer, A. S. (1996). Language testing in practice: Designing and developing useful language tests. Oxford University Press.
Chapelle, C. A., & Douglas, D. (2006). Assessing language through computer technology. Cambridge University Press. https://doi.org/10.1017/CBO9780511733116
Cushing, S. T., Ren, H., & Tan, Y. (2024). The use of TOEFL iBT® in admissions decisions: Stakeholder perceptions of policies and practices (ETS Research Report Series). Educational Testing Service. https://files.eric.ed.gov/fulltext/EJ1459649.pdf
Dermo, J. (2009). e-Assessment and the student learning experience: A survey of student perceptions of e-assessment. British Journal of Educational Technology, 40(2), 203-214. https://doi.org/10.1111/j.1467-8535.2008.00915.x
Douglas, D., & Hegelheimer, V. (2007). Assessing language using computer technology. Annual Review of Applied Linguistics, 27, 115-132. https://doi.org/10.1017/S0267190508070062
Dreher, C., Reiners, T., & Dreher, H. (2011). Investigating factors affecting the uptake of automated assessment technology. Journal of Information Technology Education: Research, 10, 161-181. https://doi.org/10.28945/1492
Elder, C., & Iwashita, N. (2005). Planning in language testing. In R. Ellis (Ed.), Planning and task performance in a second language (pp. 217-238). John Benjamins. https://doi.org/10.1075/lllt.11?locatt=mode:legacy
Galaczi, E. D. (2010). Face-to-face and computer-based assessment of speaking: Challenges and opportunities. In L. Araújo (Ed.), Proceedings of the Computer-based Assessment (CBA) of Foreign Language Speaking Skills (pp. 29-51). European Union. https://www.researchgate.net/publication/281090096_FACE-TO-FACE_AND_COMPUTER-BASED_ASSESSMENT_OF_SPEAKING_CHALLENGES_AND_OPPORTUNITIES
Guest, G., Namey, E., & McKenna, K. (2017). How many focus groups are enough? Building an evidence base for nonprobability sample sizes. Field Methods, 29(1), 3-22. https://doi.org/10.1177/1525822X16639015
Hayashi, Y., & Kondo, Y. (2024). Automated speech scoring of dialogue response by Japanese learners of English as a foreign language. Innovation in Language Learning and Teaching, 18(1), 32-46. https://doi.org/10.1080/17501229.2023.2217181
Hu, H., Gong, Q., & Mohd-Said, N. (2025). Exploring a decade of research: A systematic review of computer-based English speaking tests. Forum of Linguistic Studies, 7(4), 788-803. https://doi.org/10.30564/fls.v7i4.8978
Isbell, D. R., Crowther, D., & Nishizawa, H. (2024). Speaking performances, stakeholder perceptions, and test scores: Extrapolating from the Duolingo English Test to the university. Language Testing, 41(2), 233-257. https://doi.org/10.1177/02655322231165984
Jeong, H., Hashizume, H., Sugiura, M., Sassa, Y., Yokoyama, S., & Shiozaki, S. (2011). Testing second language oral proficiency in direct and semidirect settings: A social-cognitive neuroscience perspective. Language Learning, 61(3), 675-699. https://doi.org/10.1111/j.1467-9922.2011.00635.x
Kenyon, D. M., & Malabonga, V. (2001). Comparing examinee attitudes toward computer-assisted and other oral proficiency assessments. Language Learning & Technology, 5(2), 60-83. https://www.learntechlib.org/p/90922/
Kitzinger, J. (1995). Qualitative research: Introducing focus groups. British Medical Journal, 311(7000), 299–302. https://doi.org/10.1136/bmj.311.7000.299
Krueger, R. A., & Casey, M. A. (2000). Focus groups: A practical guide for applied research (3rd ed.). Sage Publications.
Kunnan, A. J. (2004). Test fairness. In M. Milanovic & C. Weir (Eds.), European language testing in a global context: Proceedings of the ALTE Barcelona Conference (pp. 27-48). Cambridge University Press. https://assets.cambridge.org/97805215/35878/frontmatter/9780521535878_frontmatter.pdf
Liu, J., & Zhang, B. (2020). Multi-level Rasch model analysis of computer-assisted automated scoring of English listening and speaking tests. In Proceedings of 2020 International Conference on Computer Engineering and Application (pp. 632–636). https://doi.org/10.1109/ICCEA50009.2020.00138
Morgan, D. L. (1997). Focus groups as qualitative research. Sage Publications. https://doi.org/10.4135/9781412984287
Nguyen, T. H. H., Nguyen, B. T. T., Hoang, G. T. L., Pham, N. T. H., & Dang, T. T. T. (2024). Computer-delivered vs. face-to-face score comparability and test takers’ perceptions: The case of the two English speaking proficiency tests for Vietnamese EFL learners. Language Testing in Asia, 14, 6. https://doi.org/10.1186/s40468-024-00277-1
Nirma, Sujariati, & Ariana. (2024). Focus group discussion as communicative activity in enhancing students’ speaking skill at SMAN 5 Barru. English Language Teaching Methodology, 4(3), 335-345. https://doi.org/10.56983/eltm.v4i3.473
Pizarro, M. A., & Laborda, J. G. (2017). Analyzing test-takers’ views on a computer-based speaking test. Profile: Issues in Teachers' Professional Development, 19(Suppl. 1), 23-38. http://dx.doi.org/10.15446/profile.v19n_sup1.68447
Quaid, E., & Barrett, A. (2021). Interpretations of spoken utterance fluency in simulated and face-to-face oral proficiency interviews. Language Education & Assessment, 4(1), 1-18. https://doi.org/10.29140/lea.v4n1.385
Sachdeva, S., Tamrakar, K. A., Perwez, E., Kapoor, P., & Gupta, D. (2024). Focus group discussion: An emerging qualitative tool for educational research. International Journal of Research and Review, 11(9), 302-308. https://doi.org/10.52403/ijrr.20240932
Yulianto, D., & Mujtahin, N. M. (2021). Online assessment during Covid-19 pandemic: EFL examiners’ perspectives and their practices. Journal of English Teaching, 7(2), 229-242. https://doi.org/10.33541/jet.v7i2.2770
Zhou, Y. (2015). Computer-delivered or face-to-face: Effects of delivery mode on the testing of second language speaking. Language Testing in Asia, 5(2), 1-16. https://doi.org/10.1186/s40468-014-0012-y
Zhu, W., & Flaitz, J. (2005). Using focus group methodology to understand international students’ academic language needs: a comparison of perspectives. The Electronic Journal of English as a Second Language, 8(4). https://files.eric.ed.gov/fulltext/EJ1068108.pdf