A STUDY ON ASSESSMENT CRITERIA FOR ENGLISH – VIETNAMESE CONSECUTIVE INTERPRETING TESTS
Main Article Content
Abstract
The study investigates the reliability and user feedback about the rubrics to evaluate English – Vietnamese consecutive interpreting tests taken by undergraduates at VNU-ULIS created by Tran and Do (2022). Five VNU-ULIS raters – two experienced raters and three novice ones – independently rated ten different interpreting tests and provided their feedback on the rubrics. The results reveal the newly created rubrics is mostly considered user-friendly and practical application for interpreting evaluation. Overall, inter-rater reliability, which was presented through Cronbach’s alpha and the single measure intra-class coefficient, was acceptable. Besides, the value among the novice raters was higher than that between the two experienced ones. The raters’ perception of each quality criterion and their rating process may account for the differences in their score decisions. The findings also suggest further improvements in terms of descriptor wording, weightings and rater training.
Article Details
Keywords
consecutive interpreting tests, assessment criteria, rubrics
References
Angelelli, C. V. (2009). Using a rubric to assess translation ability: Defining the construct. In C. V. Angelelli & H. E. Jacobson (Eds.), Testing and assessment in translation and interpreting studies: A call for dialogue between research and practice (pp. 13-47). John Benjamins.
Association internationale des interprètes de conférence. (1982). Practical guide for professional interpreters. International Association of Conference Interpreters.
Bontempo, K., & Hutchinson, B. (2011). Striving for an ‘A’ grade: A case study in performance management of interpreters. International Journal of Interpreter Education, 3(1), 56-71.
Bühler, H. (1986). Linguistic (semantic) and extra-linguistic (pragmatic) criteria for the evaluation of conference interpretation and interpreters. Multilingua, 5(4), 231-235.
Chiaro, D., & Nocella, G. (2004). Interpreters’ perception of linguistic and nonlinguistic factors affecting quality: A survey through the world wide web. Meta, 49(2), 278-293.
Gile, D. (1988). Le partage de l’attention et le ‘modèle d’effort’ en interprétation simultanée. The Interpreter’s Newsletter, 1, 4-22.
Gile, D. (1991). A communication-oriented analysis of quality in nonliterary translation and interpretation. In M. L. Larson (Ed.), Translation: Theory and practice. Tension and interdependence (pp. 188-200). John Benjamins.
Gile, D. (2001). Consecutive vs. simultaneous: Which is more accurate. Interpretation Studies, (1), 8-20.
Gillies, A. (2019). Consecutive interpreting: A short course. Routledge.
Glen, S. (n.d.). Cronbach’s alpha: Simple definition, use and interpretation. Statisticshowto. https://www.statisticshowto.com/probability and statistics statistics-definitions/cronbachs-alpha-spss/
Glen, S. (n.d.). Intraclass correlation. Statisticshowto. https://www.statisticshowto.com/intraclass-correlation/
Hale, S. (2007). Community interpreting. Palgrave Macmillan.
Hale, S., Garcia, I., Hlavac, J., Kim, M., Lai, M., Turner, B., & Slatyer, H. (2012). Improvements to NAATI testing report. NAATI. http://www.naati.com.au/PDF/INT/INTFinalReport.pdf
Jin, Y. (2017). Consecutive interpreting. In C. Shei & Z. M. Gao, The Routledge handbook of Chinese translation (pp. 321-335). Routledge.
Kalina, S. (2005). Quality assurance for interpreting processes. Journal des traducteurs/Meta: Translators' Journal, 50, 768-784.
Kurz, I. (1989). Conference interpreting: User expectations. In D. Hamond (Ed.), Coming of age: Proceedings of The 30th Annual Conference of The American Translators Association (pp. 143-148). Learned Information Inc.
Lee, J. (2008). Rating scales for interpreting performance assessment. The Interpreter and Translator Trainer, 2(2), 165-184.
Lee, S. B. (2015). Developing an analytic scale for assessing undergraduate students’ consecutive interpreting performances. Interpreting, 17(2), 226-254.
Lee, S. B. (2018). Scale-referenced, summative peer assessment in undergraduate interpreter training: Self-reflection from an action researcher. Educational Action Research, 27(2), 152-172.
Liu, M. (2013). Design and analysis of Taiwan’s interpretation certification examination. In D. Tsagari & R. van Deemter (Eds.), Assessment issues in language translation and interpreting (pp. 163-178). Peter Lang Edition.
Mahmoodzahed, K. (1992). Consecutive interpreting: Its principles and techniques. In C. Dollerup & A. Loddegaard (Eds.), Teaching translation and interpreting: Training talent and experience (pp. 231-236). John Benjamins.
Mariana, V., Cox, T., & Melby, A. K. (2015). The multidimensional quality metrics (MQM) framework: A new framework for translation quality assessment. The Journal of Specialised Translation, 23, 137-161.
McNamara, C. (1999). General guidelines for conducting interviews. Managementhelp.org. https://managementhelp.org/businessresearch/interviews.htm
Mesa, A. M. (1997). L’interprète culturel: Un professionel apprécié. Étude sur les services d’interprétation: Le point de vue des clients, des intervenants et des interprètes. Régie régionale de la santé et des services sociaux de Montréal-Centre.
Moser-Mercer, B. (1996). Quality in interpreting: Some methodological issues. The Interpreters’ Newsletter, 7, 43-55.
Nolan, J. (2005). Professional interpreting in the Real World series: Interpretation techniques and exercises. Linguistic services.
O’Brien, S. (2012). Towards a dynamic quality evaluation model for translation. Journal of Specialized Translation, 17, 55-77.
Ouvrard, G. (2013). L’interprétation consécutive officielle. Traduire, 229, 81-95.
Pienaar, M., & Cornelius, E. (2015). Contemporary perceptions of interpreting in South Africa. Nordic Journal of African Studies, 24(2), 186-206.
Pöchhacker, F. (2001). Quality assessment in conference and community interpreting. Meta, 46(2), 410-425. https://doi.org/10.7202/003847ar
Riccardi, A. (2002). Evaluation in interpretation: Macrocriteria and microcriteria. In E. Hung (Ed.), Teaching translation and interpreting 4: Building bridges (pp. 115-126). John Benjamins.
Roberts, R. P. (2000). Interpreter assessment tools for different settings. In R. P. Roberts, S. E. Carr, D. Abraham, & A. Dufour (Eds.), The critical link 2: Interpreters in the community (pp. 103-120). John Benjamins.
Russell, D., & Takeda, K. (2015). Consecutive interpreting. In R. Jourdenais & H. Mikkelson (Eds.), The Routledge handbook of interpreting (pp. 88-102). Routledge.
Setton, R., & Dawrant, A. (2016). Conference interpreting: A complete course. Benjamins.
Shuttleworth, M., & Cowie, M. (1997). Dictionary of translation studies. St. Jerome.
Tran, P. L., & Do, M. H. (2022, April 24). Interpreting quality assessment criteria and implications for English-Vietnamese consecutive interpreting quality assessment in educational context [Conference presentation abstract]. ULIS National Conference 2022, Hanoi, Vietnam.
Wang, J.-H., Napier, J., Goswell, D., & Carmichael, A. (2015). The design and application of rubrics to assess signed language interpreting performance. The Interpreter and Translator Trainer, 9(1), 83-103.
Wu, J., Liu, M., & Liao, C. (20l3). Analytic scoring in interpretation test: Construct validity and the halo effect. In H.-H. Liao, T.-E. Kao & Y. Lin (Eds.), The making of a translator. Multiple perspectives (pp. 277-292). Bookman.
Zwischenberger, C. (2010). Quality criteria in simultaneous interpreting: An international vs. a national view. The Interpreters' Newsletter, 15, 127-142.
Association internationale des interprètes de conférence. (1982). Practical guide for professional interpreters. International Association of Conference Interpreters.
Bontempo, K., & Hutchinson, B. (2011). Striving for an ‘A’ grade: A case study in performance management of interpreters. International Journal of Interpreter Education, 3(1), 56-71.
Bühler, H. (1986). Linguistic (semantic) and extra-linguistic (pragmatic) criteria for the evaluation of conference interpretation and interpreters. Multilingua, 5(4), 231-235.
Chiaro, D., & Nocella, G. (2004). Interpreters’ perception of linguistic and nonlinguistic factors affecting quality: A survey through the world wide web. Meta, 49(2), 278-293.
Gile, D. (1988). Le partage de l’attention et le ‘modèle d’effort’ en interprétation simultanée. The Interpreter’s Newsletter, 1, 4-22.
Gile, D. (1991). A communication-oriented analysis of quality in nonliterary translation and interpretation. In M. L. Larson (Ed.), Translation: Theory and practice. Tension and interdependence (pp. 188-200). John Benjamins.
Gile, D. (2001). Consecutive vs. simultaneous: Which is more accurate. Interpretation Studies, (1), 8-20.
Gillies, A. (2019). Consecutive interpreting: A short course. Routledge.
Glen, S. (n.d.). Cronbach’s alpha: Simple definition, use and interpretation. Statisticshowto. https://www.statisticshowto.com/probability and statistics statistics-definitions/cronbachs-alpha-spss/
Glen, S. (n.d.). Intraclass correlation. Statisticshowto. https://www.statisticshowto.com/intraclass-correlation/
Hale, S. (2007). Community interpreting. Palgrave Macmillan.
Hale, S., Garcia, I., Hlavac, J., Kim, M., Lai, M., Turner, B., & Slatyer, H. (2012). Improvements to NAATI testing report. NAATI. http://www.naati.com.au/PDF/INT/INTFinalReport.pdf
Jin, Y. (2017). Consecutive interpreting. In C. Shei & Z. M. Gao, The Routledge handbook of Chinese translation (pp. 321-335). Routledge.
Kalina, S. (2005). Quality assurance for interpreting processes. Journal des traducteurs/Meta: Translators' Journal, 50, 768-784.
Kurz, I. (1989). Conference interpreting: User expectations. In D. Hamond (Ed.), Coming of age: Proceedings of The 30th Annual Conference of The American Translators Association (pp. 143-148). Learned Information Inc.
Lee, J. (2008). Rating scales for interpreting performance assessment. The Interpreter and Translator Trainer, 2(2), 165-184.
Lee, S. B. (2015). Developing an analytic scale for assessing undergraduate students’ consecutive interpreting performances. Interpreting, 17(2), 226-254.
Lee, S. B. (2018). Scale-referenced, summative peer assessment in undergraduate interpreter training: Self-reflection from an action researcher. Educational Action Research, 27(2), 152-172.
Liu, M. (2013). Design and analysis of Taiwan’s interpretation certification examination. In D. Tsagari & R. van Deemter (Eds.), Assessment issues in language translation and interpreting (pp. 163-178). Peter Lang Edition.
Mahmoodzahed, K. (1992). Consecutive interpreting: Its principles and techniques. In C. Dollerup & A. Loddegaard (Eds.), Teaching translation and interpreting: Training talent and experience (pp. 231-236). John Benjamins.
Mariana, V., Cox, T., & Melby, A. K. (2015). The multidimensional quality metrics (MQM) framework: A new framework for translation quality assessment. The Journal of Specialised Translation, 23, 137-161.
McNamara, C. (1999). General guidelines for conducting interviews. Managementhelp.org. https://managementhelp.org/businessresearch/interviews.htm
Mesa, A. M. (1997). L’interprète culturel: Un professionel apprécié. Étude sur les services d’interprétation: Le point de vue des clients, des intervenants et des interprètes. Régie régionale de la santé et des services sociaux de Montréal-Centre.
Moser-Mercer, B. (1996). Quality in interpreting: Some methodological issues. The Interpreters’ Newsletter, 7, 43-55.
Nolan, J. (2005). Professional interpreting in the Real World series: Interpretation techniques and exercises. Linguistic services.
O’Brien, S. (2012). Towards a dynamic quality evaluation model for translation. Journal of Specialized Translation, 17, 55-77.
Ouvrard, G. (2013). L’interprétation consécutive officielle. Traduire, 229, 81-95.
Pienaar, M., & Cornelius, E. (2015). Contemporary perceptions of interpreting in South Africa. Nordic Journal of African Studies, 24(2), 186-206.
Pöchhacker, F. (2001). Quality assessment in conference and community interpreting. Meta, 46(2), 410-425. https://doi.org/10.7202/003847ar
Riccardi, A. (2002). Evaluation in interpretation: Macrocriteria and microcriteria. In E. Hung (Ed.), Teaching translation and interpreting 4: Building bridges (pp. 115-126). John Benjamins.
Roberts, R. P. (2000). Interpreter assessment tools for different settings. In R. P. Roberts, S. E. Carr, D. Abraham, & A. Dufour (Eds.), The critical link 2: Interpreters in the community (pp. 103-120). John Benjamins.
Russell, D., & Takeda, K. (2015). Consecutive interpreting. In R. Jourdenais & H. Mikkelson (Eds.), The Routledge handbook of interpreting (pp. 88-102). Routledge.
Setton, R., & Dawrant, A. (2016). Conference interpreting: A complete course. Benjamins.
Shuttleworth, M., & Cowie, M. (1997). Dictionary of translation studies. St. Jerome.
Tran, P. L., & Do, M. H. (2022, April 24). Interpreting quality assessment criteria and implications for English-Vietnamese consecutive interpreting quality assessment in educational context [Conference presentation abstract]. ULIS National Conference 2022, Hanoi, Vietnam.
Wang, J.-H., Napier, J., Goswell, D., & Carmichael, A. (2015). The design and application of rubrics to assess signed language interpreting performance. The Interpreter and Translator Trainer, 9(1), 83-103.
Wu, J., Liu, M., & Liao, C. (20l3). Analytic scoring in interpretation test: Construct validity and the halo effect. In H.-H. Liao, T.-E. Kao & Y. Lin (Eds.), The making of a translator. Multiple perspectives (pp. 277-292). Bookman.
Zwischenberger, C. (2010). Quality criteria in simultaneous interpreting: An international vs. a national view. The Interpreters' Newsletter, 15, 127-142.