Abstract
The study of pragmatic competence has gained increasing importance within second language assessment over the last three decades. However, its study in L2 language testing is still scarce. The aim of this paper is to research the extent to which pragmatic competence as defined by the Common European Framework of Reference for Languages (CEFR) has been accommodated in the task descriptions and rating scales of two of the most popular Oral Proficiency Interviews (OPIs) at a C1 level: Cambridge’s Certificate in Advanced English (CAE) and Trinity’s Integrated Skills in English (ISE) III. To carry out this research, OPI tests are first defined, highlighting their differences from L2 pragmatic tests. After pragmatic competence in the CEFR is examined, focusing on the updates in the new descriptors, CAE and ISE III formats, structure and task characteristics are compared, showing that, while the formats and some characteristics are found to differ, the structures and task types are comparable. Finally, we systematically analyse CEFR pragmatic competence in the task skills and rating scale descriptors of both OPIs. The findings show that the task descriptions incorporate mostly aspects of discourse and design competence. Additionally, we find that each OPI is seen to prioritise different aspects of pragmatic competence within their rating scale, with CAE focusing mostly on discourse competence and fluency, and ISE III on functional competence. Our study shows that the tests fail to fully accommodate all aspects of pragmatic competence in the task skills and rating scales, although the aspects they do incorporate follow the CEFR descriptors on pragmatic competence. It also reveals a mismatch between the task competences being tested and the rating scale. To conclude, some research lines are proposed.
About the authors
Bárbara Eizaga-Rebollar is a tenured professor in the Department of French and English Studies at the University of Cádiz (Spain) and Member of the Research Centre for Applied Linguistics at same university. Her main research interests focus on pragmatics, cognitive linguistics, applied linguistics and language assessment. She has published internationally in linguistic journals and volumes (L’Harmattan, Peter Lang, Procedia, Spanish in Context or Ibérica, among others).
Cristina Heras-Ramírez is a PhD student at the University of Cádiz. Her research interests revolve around the field of applied linguistics, English as a second language, pragmatics and language assessment. She holds a Bachelor’s degree in English studies from the University of Cadiz and a Master’s degree in the Teaching of Languages from the University of Southern Mississippi, where she was awarded a graduate assistantship. Furthermore, she worked as technical support staff at the Research Centre for Applied Linguistics at the University of Cádiz and has participated in international conferences and science outreach activities.
References
Bachman, Lyle. 1990. Fundamental considerations in language testing New York: Oxford University Press.Search in Google Scholar
Bachman, Lyle & Adrian Palmer. 1996. Language Testing in Practice: Designing and Developing Useful Language Tests Oxford: Oxford University Press.Search in Google Scholar
Blommaert, Jan & Jef Verschueren (eds.). 1991. The Pragmatics of Intercultural and International Communication. Selected papers of the International Pragmatics Conference, Antwerp, August 17–22, 1987 (Volume III), and the Ghent Symposium on Intercultural Communication Amsterdam/Philadelphia: John Benjamins Publishing Company.10.1075/pbns.6.3Search in Google Scholar
Blum-Kulka, Shoshana, Juliane House & Gabriele Kasper (eds.). 1989. Cross-cultural pragmatics: Requests and apologies Norwood, NJ: Ablex.Search in Google Scholar
Bosker, Hans Rutger, Anne-France Pinget, Hugo Quené, Ted Sanders & Nivja H. de Jong. 2013. What makes speech sound fluent? The contributions of pauses, speed and repairs. Language Testing 30(2). 159–175.10.1177/0265532212455394Search in Google Scholar
Brooks, Lindsay. 2009. Interacting in pairs in a test of oral proficiency: Co-constructing a better performance. Language Testing 26(3). 341–366.10.1177/0265532209104666Search in Google Scholar
Brown, Annie. 2003. Interviewer variation and the co-construction of speaking proficiency. Language Testing 20(1). 1–25.10.1191/0265532203lt242oaSearch in Google Scholar
Brown, Penelope & Stephen Levinson. 1987. Politeness: Some universals in language usage Cambridge: Cambridge University Press.10.1017/CBO9780511813085Search in Google Scholar
Cambridge Assessment English (CAE). 2012. Assessment commentary and marks. Commentary on CAE Speaking test: Meritxell and Stefan. Available at: http://readeralexey.narod.ru/ENGLISH/marks_and_commentary_merixtell_stefan.pdf (accessed 11 June, 2020).Search in Google Scholar
Cambridge Assessment English (CAE). 2015. Cambridge English: Advanced (2015 Update) Assessment Commentary and Marks: Raphael and Maude Available at: https://www.cambridgeenglish.org/Images/167862-cambridge-english-advanced-cae-from-2015-speaking-test-video.pdf (accessed 11 June, 2020).Search in Google Scholar
Cambridge Assessment English (CAE). 2019. Cambridge English Qualifications C1 Advanced. Handbook for teachers. Available at: https://www.cambridgeenglish.org/Images/167804-cambridge-english-advanced-handbook.pdf (accessed 30 September ,2019).Search in Google Scholar
Canale, Michael. 1983. From communicative competence to communicative language pedagogy. In Jack C. Richards & Richard W. Schmidt (eds.). Language and Communication, 1–27. London and New York: Routledge.Search in Google Scholar
Canale, Michael and Merrill Swain. 1980. Theoretical bases of communicative approaches to second language teaching and testing. Applied Linguistics 1. 1-47.10.1093/applin/1.1.1Search in Google Scholar
Connor-Linton, Jeff & Elana Shohamy. 2001. Register variation, oral proficiency sampling, and the promise of multi-dimensional analysis. In Susan Conrad & Douglas Biber (eds.), Variation in English: Multi-Dimensional Studies 124–137. London and New York: Routledge.Search in Google Scholar
Council of Europe (COE). 2001. Common European Framework of Reference for Languages: Learning, teaching, assessment. Cambridge: Cambridge University Press.Search in Google Scholar
Council of Europe (COE). 2018. The CEFR Companion Volume with New Descriptors. Language Policy Programme, Education Policy Division, Education Department. Available at: https://rm.coe.int/cefr-companion-volume-with-new-descriptors-2018/1680787989 (accessed 30 September, 2019).Search in Google Scholar
De Jong, Nivja H. 2017. Fluency in second language assessment. In Dina Tsagari & Jayanti Banerjee (eds.), Handbook of Second Language Assessment 203–218. Boston/Berlin: De Gruyter Mouton.10.1515/9781614513827-015Search in Google Scholar
Ficzere, Edit. 2019. An investigation into assessing ESL learners’ pragmatic competence at B2-C2 levels. PhD. Available at: https://uobrep.openrepository.com/handle/10547/623986 (accessed 2 April, 2020).Search in Google Scholar
Field, John. 2011. Cognitive validity. In Lynda Taylor (ed.), Studies in Language Testing. Examining Speaking 65–111. Cambridge: Cambridge University Press.Search in Google Scholar
Fotos, Sandra & Rod Ellis. 1999. Communicating About Grammar. In Rod Ellis (ed.), Learning a Second Language Through Interaction 189–208. Amsterdam/Philadelphia: John Benjamins Publishing Company.10.1075/sibil.17.13fotSearch in Google Scholar
Fulcher, Glenn. 2003. Testing Second Language Speaking. London and New York: Routledge.Search in Google Scholar
Galaczi, Evelina. 2014. Interactional competence across proficiency levels: How do learners manage interaction in paired speaking tests? Applied Linguistics 35(5). 553–574.10.1093/applin/amt017Search in Google Scholar
Galaczi, Evelina & Lynda Taylor. 2018. Interactional competence: Conceptualisations, Operationalisations, and Outstanding Questions. Language Assessment Quarterly 115(3). 219–236.10.1080/15434303.2018.1453816Search in Google Scholar
House, Juliane. 2002. Developing pragmatic competence in English as a lingua franca. In Knapp Karlfried & Christiane Meierkord (eds.), Lingua Franca Communication, 245–267. Frankfurt am Main: Peter Lang.Search in Google Scholar
Housen, Alex, Fokert Kuiken & Ineke Vedder. 2012. Complexity, accuracy and fluency. In Alex Housen, Fokert Kuiken & Ineke Vedder (eds.), Dimensions of L2 Performance and Proficiency. Complexity, Accuracy and Fluency in SLA 1–20. Amsterdam/Philadelphia: John Benjamins Publishing Company.10.1075/lllt.32Search in Google Scholar
Ifantidou, Elly. 2013. Pragmatic competence and explicit instruction. Journal of Pragmatics 59. 93–116.10.1016/j.pragma.2012.12.008Search in Google Scholar
Ifantidou, Elly & Angeliki Tzanne. 2012. Levels of pragmatic competence in an EFL academic context: A tool for assessment. Intercultural Pragmatics 9(1). 47–70.10.1515/ip-2012-0003Search in Google Scholar
Integrated Skills in English (ISE). 2017 [2015]. Guide for Teachers – ISE III (C1). Trinity College London. Available at: https://excellenceineducation.eu/moodle/pluginfile.php/810/mod_folder/content/0/ISE%20Specifications%20-%20Reading%20%20Writing%20-%20Third%20Edition.pdf?forcedownload=1 (accessed 27 September, 2019).Search in Google Scholar
Ishihara, Noriko. 2013. Teacher-based assessment of L2 Japanese pragmatics: Classroom applications. In Steven Ross & Gabriele Kasper (eds.), Assessing Second Language Pragmatics, 124–148. Basingstoke, UK: Palgrave Macmillan.10.1057/9781137003522_5Search in Google Scholar
Johnson, Marysia & Andrea Tyler. 1998. How Much Does It Look Like Natural Conversation? In Richard Young & Agnes Weiyun He (eds.), Talking and Testing. Discourse approaches to the assessment of oral proficiency 27–51. Amsterdam/Philadelphia: John Benjamins Publishing Company.10.1075/sibil.14.04johSearch in Google Scholar
Kasper, Gabriele, & Schmidt, Richard. 1996. Developmental issues in interlanguage pragmatics. Studies in Second Language Acquisition 18. 149–169.10.1017/S0272263100014868Search in Google Scholar
Kasper, Gabriele & Steven J. Ross. 2007. Multiple questions in oral proficiency interviews. Journal of Pragmatics 39(11). 2045–2070.10.1016/j.pragma.2007.07.011Search in Google Scholar
Kasper, Gabriele & Steven J. Ross. 2013. Overview and introduction. In Gabriele Kasper & Steven J. Ross (eds.), Assessing second language pragmatic, 1–40. New York: Palgrave MacMillan.10.1057/9781137003522Search in Google Scholar
Kecskes, Istvan. 2004. Lexical merging, conceptual blending, and cultural crossing. Intercultural Pragmatics 1(1). 1–26.10.1515/iprg.2004.005Search in Google Scholar
Kecskes, Istvan. 2014. Intercultural Pragmatics. New York, USA: Oxford University Press.10.1093/acprof:oso/9780199892655.001.0001Search in Google Scholar
Liu, Jianda. 2006. Measuring Pragmatic Knowledge: Issues of Construct Underrepresentation or Labeling? Frankfurt am Main: Peter Lang.Search in Google Scholar
Luoma, Sari. 2004. Assessing Speaking. Cambridge: Cambridge University Press.10.1017/CBO9780511733017Search in Google Scholar
McNamara, Tim & Carsten Roever. 2006. Language Testing: The Social Dimension. New Jersey: John Wiley and Sons.Search in Google Scholar
Okada, Yusuke. 2010. Role-play in oral proficiency interviews: Interactive footing and interactional competencies. Journal of Pragmatics, 42. 1674–1668.10.1016/j.pragma.2009.11.002Search in Google Scholar
O’Loughlin, Kieran. 2001. The Equivalence of Direct and Semi-Direct Speaking Tests Cambridge: Cambridge University Press.Search in Google Scholar
Piccardo, Enrica & Brian North. 2019. The Action-oriented Approach: A Dynamic Vision of Language Education Bristol: Multilingual Matters.10.21832/9781788924351Search in Google Scholar
Robinson, Peter. 2001. Task complexity, cognitive resources, and syllabus design: a triadic framework for examining task influences on SLA. In Peter Robinson (ed.), Cognition and Second Language Instruction 287-318. Cambridge: Cambridge University Press.10.1017/CBO9781139524780Search in Google Scholar
Roever, Carsten. 2006. Validation of a web-based test of ESL pragmalinguistics. Language Testing 23(2). 229–256.10.1191/0265532206lt329oaSearch in Google Scholar
Roever, Carsten. 2011. Testing of second language pragmatics: Past and future. Language Testing 28(4). 463–481.10.1177/0265532210394633Search in Google Scholar
Ross, Steven. 2007. A comparative task-in-interaction analysis of OPI backsliding. Journal of Pragmatics 39(11). 2017–2044.10.1016/j.pragma.2007.07.010Search in Google Scholar
Ross, Steven & Gabriele Kasper. 2013. Assessing second language pragmatics: An overview and introductions. In Steven Ross & Gabriele Kasper (eds.), Assessing Second Language Pragmatics, 1–40. Basingstoke, UK: Palgrave Macmillan.10.1057/9781137003522Search in Google Scholar
Schneider, Klaus P. 2008. Small talk in England, Ireland, and the U.S.A. In Klaus P. Schneider & Anne Barron, Variational Pragmatics: a focus on regional varieties in pluricentric languages 99–139. Amsterdam/Philadelphia: John Benjamins Publishing Company.10.1075/pbns.178.06schSearch in Google Scholar
Searle, John. R. 1969. Speech Acts: An essay in the philosophy of language Cambridge: Cambridge University Press.10.1017/CBO9781139173438Search in Google Scholar
Seedhouse, Paul. 2013. Oral proficiency interviews as varieties of interaction. In Steven Ross & Gabriele Kasper (eds.), Assessing Second Language Pragmatics, 199–219. Basingstoke, UK: Palgrave Macmillan.10.1057/9781137003522_8Search in Google Scholar
Staples, Shelley, Geoffrey T. Lafflair & Jesse Edbert. 2017. Comparing Language Use in Oral Proficiency Interviews to Target Domains: Conversational, Academic, and Professional Discourse. The Modern Language Journal 101(1). 194–213.10.1111/modl.12385Search in Google Scholar
Taguchi, Naoko. 2009. Pragmatic competence in Japanese as a second language: An introduction. In Naoko Taguchi (ed.), Pragmatic Competence 1–18. Berlin, New York: Mouton de Gruyter.10.1515/9783110218558Search in Google Scholar
Taguchi, Naoko. 2013. Refusals in L2 English: Proficiency effects on appropriateness and fluency. In Patricia Salazar-Campillo & Otilia Martí-Armándiz (eds.), Refusals in instructional contexts and beyond, 101–120. Spain: Rodopi.10.1163/9789401209717_007Search in Google Scholar
Taguchi, Naoko. 2017. Interlanguage pragmatics: A historical sketch and future directions. In Anne Barron, Gu Yuego & Gerard Steen (eds.), The Routledge Handbook of Pragmatics 153–167. Oxford/NewYork: Routledge.10.4324/9781315668925-14Search in Google Scholar
Takač, Višnja Pavičić & Vesna Bagarić Medve. 2016. The Functional Competence of EFL learners. In Irena Zovko Dinković & Jelena Mihaljević Djigunović (eds.), English Studies from Archives to Prospects. Volume 2 - Linguistics and Applied Linguistics 234–253. Newcastle upon Tyne: Cambridge Scholars Publishing.Search in Google Scholar
Tominaga, Waka. 2013. The development of extended turns and storytelling in the Japanese oral proficiency interview. In Steven Ross & Gabriele Kasper (eds.), Assessing Second Language Pragmatics, 220–257. Basingstoke, UK: Palgrave Macmillan.10.1057/9781137003522_9Search in Google Scholar
Trinity College London. 2020. Videos - ISE III (C1) Available at: https://www.trinitycollege.com/qualifications/english-language/ISE/ISE-III-C1-resources/ISE-III-C1-videos (accessed 15 June, 2020).Search in Google Scholar
Wilson, Deirdre & Dan Sperber. 2004. Relevance Theory. In Laurence R. Horn & Gregory Ward (eds.), The Handbook of Pragmatics, 607–632. Oxford: Blackwell Publishing.Search in Google Scholar
Youn, Soo Jung. 2015. Validity argument for assessing L2 pragmatics in interaction using mix methods. Language Testing 32(2). 199–225.10.1177/0265532214557113Search in Google Scholar
Young, Richard Frederick. 2011. Interactional competence in language learning, teaching, and testing. In Eli Hinkel (ed.), Handbook of Research in Second Language Teaching and Learning Vol. 1, 426–433. New York: Routledge.Search in Google Scholar
Young, Richard Frederick. 2019. Interactional Competence and L2 Pragmatics. The Routledge handbook of second language acquisition and pragmatics, 93–110. New York: Routledge.10.4324/9781351164085-7Search in Google Scholar
© 2020 Walter de Gruyter GmbH, Berlin/Boston
