DEFINING AN ALTERNATIVE ASSESSMENT CONSTRUCT AS A RATING SCALE FOR EVALUATION OF LSP ORAL PRESENTATION
Keywords:
construct, alternative assessment, validity, rating scale
Abstract
The subject of the research is defining an oral presentation assessment construct in language for specific purposes. The aim of the research bases on defining validity of an alternative form of assessment, such as performance, in the manner of the standardised testing procedure. More precisely, starting form the students’ performances, i.e. oral presentations in language for specific purposes that needs to be evaluated, we analysed theoretical frameworks of communicative competence and specific purpose language ability, as well as speech parameters and public speaking rating scales’ characteristics. This approach was adopted in order to reach an optimal solution for an assessment construct of oral presentation performance. As the theoretical background shows that an assessment construct can be operationalised by rating scale criteria, we designed a unique rating scale for oral presentation assessment based on the results of the theoretical analysis. Thus, the rating scale for oral presentation assessment is composed of the parameters of linguistic, sociolinguistic, discourse and functional competences as well as on performance strategies and background knowledge. The main purpose of such rater-oriented scale is to increase objectivity and a priori validity of assessment procedure in the real classroom context.
References
Alderson, J. C. (1991). Bands and Scores. In J. C. Alderson & B. North (eds.): Language Testing in the 1990s, 71-86. London: Macmillan.
Ashford-Rowe, K., Herrington, J. & Brown, C. (2014). Establishing the critical elements that determine authentic assessment, Assessment & Evaluation in Higher Education, 39(2), 205-222. http://dx.doi.org/10.1080/02602938.2013.819566
Bachman, L. F. & Palmer, A. S. (1996). Language testing in practice: Designing and developing useful language tests (Vol. 1), Oxford: Oxford University Press.
Bachman, L. F. (2000). Modern language testing at the turn of the century: assuring that what we count counts, Language testing, 17(1), 1-42.
Boud, D., & Falchikov, N. (2005). Redesigning assessment for learning beyond higher education. Research and development in higher education, 28 (special issue), 34-41. https://conference.herdsa.org.au/2005/pdf/refereed/paper_398.pdf
Brown, J. D. & Hudson, T. (1998). The alternatives in language assessment, TESOL Quarterly, 32(4), 653-675.
Chan, S. & Latimer, N. (2020). Placing construct definition at the heart of assessment: research, design and a priori validation. In Taylor, L. & Saville, N. (eds.): Lessons and Legacy: A Tribute to Professor Cyril J. Weir (1950–2018), (pp. 105-132). Cambridge: Cambridge University Press 2020. https://www.cambridgeenglish.org/Images/582822-silt-volume-50.pdf#page=129
Chen, G. (2016). Developing a Model of Analytic Rating Scales to Assess College Students’ L2 Chinese Oral Performance, International Journal of Language Testing, 6 (2), 50-71.
Clapham, C. (2000). Assessment and Testing. Annual Review of Applied Linguistics, 20, 147-161.
Council of Europe. (2001). Common European framework of Reference for Languages: Learning, Teaching, Assessment, Cambridge: Cambridge University Press.
Dimitrijević, N. (1999). Testiranje u nastavi stranih jezika, Beograd: Zavod za udžbenike i nastavna sredstva.
Douglas, D. (2000). Assessing Language for Specific Purposes, Cambridge: Cambridge University Press.
Fairbairn, J. & Dunlea, J. (2017). Speaking and Writing Rating Scales Revision, Aptis Technical Report, TR/2017/001, 1-36.
Fulcher, G. (2014). Testing Second Language Speaking, London and New York: Routledge Taylor and Francis Group.
Fulcher, G., Davidson, F. & Kemp, J. (2011) Effective rating scale development for speaking tests: Performance decision trees. Language Testing, 28(1), 5-29.
Herman, J. L., Aschbacher, P.R. & Winters, L. (1992). A practical guide to alternative assessment. Alexandria, VA: Association for Supervision and Curriculum Development.
Hill, M. & Storey, A. (2003). Speak Easy: online support for oral presentation skills. ELT Journal, 57(4), 370-376. https://maaz.ihmc.us/rid=1K7F73WTZ-1LYSH40-V0B/Hill%20-%20SpeakEasy.pdf
Iberri-Shea, G. (2017). Adaptation and assessment of a public speaking rating scale, Cogent Education, 4 (1), 1-16. http://dx.doi.org/10.1080/2331186X.2017.1287390
Janković, N., Tomović, N., & Aleksić, M. (2020). Principi i kriterijumi ocenjivanja kao osnove vrednovanja postignuća u obrazovanju. U Plovina, V. i Kovačević, B. (ur.): Primenjena lingvistika danas-jezik, književnost i interdisciplinarnost. Vol. 6, str. 123-139. Beograd: Filološki fakultet, Univerzitet u Beogradu. https://doi.org/10.18485/dpls_pld.2020.6.ch7
Kavaliauskienė, G., Anusienė, L., & Kaminskienė, L. (2007). The challenges for ESP learners: alternative assessment of performance and usefulness of class activities. Social Inquiry into Well-Being, 6(1), 134-144.
Koh, K. H., Tan, C. & Ng, P. T. (2012). Creating thinking schools through authentic assessment: the case in Singapore. Educational Assessment Evaluation and Accreditation, 24, 135-149. https://link.springer.com/article/10.1007/s11092-011-9138-y
Lavinia, N., & Alina, B. T. G. (2018). Innovative Methods for Assessing Students’ Performance in the ESP Classroom. Ovidius University Annals, Economic Sciences Series, 18(1), 200-205. https://stec.univ-ovidius.ro/html/anale/RO/wp-content/uploads/2018/08/15.pdf
Luoma, S. 2004. Assessing speaking, Cambridge: Cambridge University Press.
Maclellan, E. (2004). How convincing is alternative assessment for use in higher education?. Assessment & Evaluation in Higher Education, 29(3), 311-321.
Makrieva, I. (2017). The changing attitude towards oral presentations in the ESP classroom. U Kurteva, E., Belova M. & Georgieva, I. (eds.): Foreign Languages and the contemporary higher education Conference Proceedings VIII International Scientific Conference, 23-25 June 2017, Varna (pp. 398-404). Varna: Medical University of Varna.
Morita, N. (2000). Discourse Socialization through Oral Classroom Activities in a TESL Graduate Program, TESOL Quarterly, 34 (2), 279-310. http://www.jstor.org/stable/3587953
Norris, Ј. М., Brown, J.D., Hudson, T. & Yoshioka, J. (1998). Designing second language performance assessments, Second Language Teaching & Curriculum Center, Mānoa: University of Hawai`i.
North, B. (2003). Scales for Rating Language Performance: descriptive models, formulation styles, and presentation formats, TOEFL Monograph, 24, 1-96.
Petrović, V. (2022). Evaluacija i unapređivanje kriterijuma za ocenjivanje usmene prezentacije na engleskom jeziku struke. (Neobjavljena doktorska disertacija). Filološki fakultet, Univerzitet u Beogradu.
Petrović, V. M., Jovanović, A. S., & Đukić, M. R. (2024). Developing Multicriteria Rating Scale for Assessment of Communicative Competence in LSP Oral Presentation. Teaching Innovations/Inovacije u Nastavi, 37(4), 64-78. DOI: 10.5937/inovacije2404064P
Petrović, V., Radović Firat, A. & Palurović, L. (2022). Oral presentation as authentic material for providing ESP instruction. In I. Milićević (ed.), 9th International Scientific Conference Technics and Informatics in Education – TIE 2022, 16th - 18th September 2022 Čačak (pp. 414−19): Čačak: Faculty of Technical Sciences Čačak, University of Kragujevac. DOI: 10.46793/TIE22.414P
Richards, J. C. (2008). Teaching Listening and Speaking: From Theory to Practice, Cambridge: Cambridge University Press.
Safarzadeh, M., & Taghipour Bazargani, D. (2023). Exploring Iranian ESP Teachers’ Assessment Practices in Online Specialized English Courses. International Journal of Research in English Education, 8(4), 51-60. https://ijreeonline.com/files/site1/user_files_68bcd6/bazargani74-A-10-938-3-25b4871.pdf
Savet Evrope. (2002). Zajednički evropski okvir za učenje, nastavu i ocjenjivanje jezika, Подгорица: Министарство просвијете и науке.
Vukasović, M., Turajlić, S. i Vukasović, M. (2006). Razvoj kurikuluma u visokom obrazovanju, Beograd: Alternativna akademska obrazovna mreža.
Weir, C. J. (2005). Language Testing and Validation: An Evidence-based Approach, Houndmills, Basingstoke, Hampshire, New York: Palgrave Macmillan.
Ashford-Rowe, K., Herrington, J. & Brown, C. (2014). Establishing the critical elements that determine authentic assessment, Assessment & Evaluation in Higher Education, 39(2), 205-222. http://dx.doi.org/10.1080/02602938.2013.819566
Bachman, L. F. & Palmer, A. S. (1996). Language testing in practice: Designing and developing useful language tests (Vol. 1), Oxford: Oxford University Press.
Bachman, L. F. (2000). Modern language testing at the turn of the century: assuring that what we count counts, Language testing, 17(1), 1-42.
Boud, D., & Falchikov, N. (2005). Redesigning assessment for learning beyond higher education. Research and development in higher education, 28 (special issue), 34-41. https://conference.herdsa.org.au/2005/pdf/refereed/paper_398.pdf
Brown, J. D. & Hudson, T. (1998). The alternatives in language assessment, TESOL Quarterly, 32(4), 653-675.
Chan, S. & Latimer, N. (2020). Placing construct definition at the heart of assessment: research, design and a priori validation. In Taylor, L. & Saville, N. (eds.): Lessons and Legacy: A Tribute to Professor Cyril J. Weir (1950–2018), (pp. 105-132). Cambridge: Cambridge University Press 2020. https://www.cambridgeenglish.org/Images/582822-silt-volume-50.pdf#page=129
Chen, G. (2016). Developing a Model of Analytic Rating Scales to Assess College Students’ L2 Chinese Oral Performance, International Journal of Language Testing, 6 (2), 50-71.
Clapham, C. (2000). Assessment and Testing. Annual Review of Applied Linguistics, 20, 147-161.
Council of Europe. (2001). Common European framework of Reference for Languages: Learning, Teaching, Assessment, Cambridge: Cambridge University Press.
Dimitrijević, N. (1999). Testiranje u nastavi stranih jezika, Beograd: Zavod za udžbenike i nastavna sredstva.
Douglas, D. (2000). Assessing Language for Specific Purposes, Cambridge: Cambridge University Press.
Fairbairn, J. & Dunlea, J. (2017). Speaking and Writing Rating Scales Revision, Aptis Technical Report, TR/2017/001, 1-36.
Fulcher, G. (2014). Testing Second Language Speaking, London and New York: Routledge Taylor and Francis Group.
Fulcher, G., Davidson, F. & Kemp, J. (2011) Effective rating scale development for speaking tests: Performance decision trees. Language Testing, 28(1), 5-29.
Herman, J. L., Aschbacher, P.R. & Winters, L. (1992). A practical guide to alternative assessment. Alexandria, VA: Association for Supervision and Curriculum Development.
Hill, M. & Storey, A. (2003). Speak Easy: online support for oral presentation skills. ELT Journal, 57(4), 370-376. https://maaz.ihmc.us/rid=1K7F73WTZ-1LYSH40-V0B/Hill%20-%20SpeakEasy.pdf
Iberri-Shea, G. (2017). Adaptation and assessment of a public speaking rating scale, Cogent Education, 4 (1), 1-16. http://dx.doi.org/10.1080/2331186X.2017.1287390
Janković, N., Tomović, N., & Aleksić, M. (2020). Principi i kriterijumi ocenjivanja kao osnove vrednovanja postignuća u obrazovanju. U Plovina, V. i Kovačević, B. (ur.): Primenjena lingvistika danas-jezik, književnost i interdisciplinarnost. Vol. 6, str. 123-139. Beograd: Filološki fakultet, Univerzitet u Beogradu. https://doi.org/10.18485/dpls_pld.2020.6.ch7
Kavaliauskienė, G., Anusienė, L., & Kaminskienė, L. (2007). The challenges for ESP learners: alternative assessment of performance and usefulness of class activities. Social Inquiry into Well-Being, 6(1), 134-144.
Koh, K. H., Tan, C. & Ng, P. T. (2012). Creating thinking schools through authentic assessment: the case in Singapore. Educational Assessment Evaluation and Accreditation, 24, 135-149. https://link.springer.com/article/10.1007/s11092-011-9138-y
Lavinia, N., & Alina, B. T. G. (2018). Innovative Methods for Assessing Students’ Performance in the ESP Classroom. Ovidius University Annals, Economic Sciences Series, 18(1), 200-205. https://stec.univ-ovidius.ro/html/anale/RO/wp-content/uploads/2018/08/15.pdf
Luoma, S. 2004. Assessing speaking, Cambridge: Cambridge University Press.
Maclellan, E. (2004). How convincing is alternative assessment for use in higher education?. Assessment & Evaluation in Higher Education, 29(3), 311-321.
Makrieva, I. (2017). The changing attitude towards oral presentations in the ESP classroom. U Kurteva, E., Belova M. & Georgieva, I. (eds.): Foreign Languages and the contemporary higher education Conference Proceedings VIII International Scientific Conference, 23-25 June 2017, Varna (pp. 398-404). Varna: Medical University of Varna.
Morita, N. (2000). Discourse Socialization through Oral Classroom Activities in a TESL Graduate Program, TESOL Quarterly, 34 (2), 279-310. http://www.jstor.org/stable/3587953
Norris, Ј. М., Brown, J.D., Hudson, T. & Yoshioka, J. (1998). Designing second language performance assessments, Second Language Teaching & Curriculum Center, Mānoa: University of Hawai`i.
North, B. (2003). Scales for Rating Language Performance: descriptive models, formulation styles, and presentation formats, TOEFL Monograph, 24, 1-96.
Petrović, V. (2022). Evaluacija i unapređivanje kriterijuma za ocenjivanje usmene prezentacije na engleskom jeziku struke. (Neobjavljena doktorska disertacija). Filološki fakultet, Univerzitet u Beogradu.
Petrović, V. M., Jovanović, A. S., & Đukić, M. R. (2024). Developing Multicriteria Rating Scale for Assessment of Communicative Competence in LSP Oral Presentation. Teaching Innovations/Inovacije u Nastavi, 37(4), 64-78. DOI: 10.5937/inovacije2404064P
Petrović, V., Radović Firat, A. & Palurović, L. (2022). Oral presentation as authentic material for providing ESP instruction. In I. Milićević (ed.), 9th International Scientific Conference Technics and Informatics in Education – TIE 2022, 16th - 18th September 2022 Čačak (pp. 414−19): Čačak: Faculty of Technical Sciences Čačak, University of Kragujevac. DOI: 10.46793/TIE22.414P
Richards, J. C. (2008). Teaching Listening and Speaking: From Theory to Practice, Cambridge: Cambridge University Press.
Safarzadeh, M., & Taghipour Bazargani, D. (2023). Exploring Iranian ESP Teachers’ Assessment Practices in Online Specialized English Courses. International Journal of Research in English Education, 8(4), 51-60. https://ijreeonline.com/files/site1/user_files_68bcd6/bazargani74-A-10-938-3-25b4871.pdf
Savet Evrope. (2002). Zajednički evropski okvir za učenje, nastavu i ocjenjivanje jezika, Подгорица: Министарство просвијете и науке.
Vukasović, M., Turajlić, S. i Vukasović, M. (2006). Razvoj kurikuluma u visokom obrazovanju, Beograd: Alternativna akademska obrazovna mreža.
Weir, C. J. (2005). Language Testing and Validation: An Evidence-based Approach, Houndmills, Basingstoke, Hampshire, New York: Palgrave Macmillan.
Published
2026/03/02
Section
Članci