Abstract
This paper explores, through the use of the observation checklist and Conversation Analysis (CA), the discourse functions elicited by the dialogic task of the Free University of Bolzano (IT) speaking test. It aims to contribute to content validation, which has been claimed to be especially relevant in paired speaking tests, where interaction is co-constructed by participants. Lazaraton has argued that the investigation of the process of the assessment of speaking competences represents, for todays’ research, a question of almost the same importance as outcome scores. The paper investigates the predictions of the construct through the actual elicitation of functions in the task performance of the test. Results are analysed qualitatively with respect to informational, interactional, and management-of-interaction functions. Some quantitative analysis was also conducted to determine the relative frequency of each function. Outcomes were found to be in line with previous research in the proportion of use of the three different types of functions, informational, interactional and management of interaction. The paper finally discusses the advantages and disadvantages of exploring test content through the lenses of observation checklists and CA, the first providing a general framework in which discourse functions can be outlined, and the second providing a more fine-tuned view of the data and of the complexity of exploitation of each function.
About the author
Cecilia Varcasia currently teaches language teaching methodology at the Free University of Bolzano and language acquisition at the University of Cagliari. Her research interests lie in language testing, especially of speaking, and conversation analysis, cross-cultural pragmatics and multilingual communication.
Appendix A: Observation checklist adapted from O’Sullivan et al. (2002)
| Informational functions | |
|---|---|
| Providing personal information | – Give information on present circumstances |
| – Give information on past experiences | |
| – Give information on future plans | |
| Expressing preferences/ opinions | Express preferences/opinions |
| Elaborating/ Justifying opinions | Elaborate on, or modify an opinion Express reasons for assertions s/he had made |
| Comparing | Compare things/people/events |
| Speculating | Speculate |
| Staging | Separate out or interpret the parts of an issue |
| Describing | – Describe a sequence of events |
| – Describe a scene | |
| Summarizing | Summarize what s/he has said |
| Suggesting | Suggest a particular idea |
| Interactional functions | |
| Agreeing /Disagreeing | Agree/disagree with an assertion made by another speaker (apart from “yeah”/“no” or nonverbal) |
| Modifying | Modify arguments or comments made by other speaker or by the test-taker in response to another speaker |
| Asking for opinions | Ask for opinions |
| Persuading | Attempt to persuade another person |
| Asking for information | Ask for information |
| Conversational repair | Repair breakdowns in interaction |
| Negotiating meaning | – Check understanding |
| – Indicate understanding of point made by partner | |
| – Establish common ground/purpose or strategy | |
| – Ask for clarification when an utterance is misheard or misinterpreted | |
| – Correct an utterance made by other speaker which is perceived to be incorrect or inaccurate | |
| – Respond to requests for clarification | |
| Managing interaction | |
| Initiating | Start any interactions |
| Changing | Take the opportunity to change the topic |
| Reciprocating | Share the responsibility for developing the interaction |
| Deciding | Come to a decision |
Appendix B: Rubrics and Prompt card models
Rubrics for the dialogic task
Part 3 (5 mins [2])
Now, in this part of the test you’ll be speaking to each other, discussing a topic together and expressing your opinions.
Interlocutor shows card and reads task instructions from card
Hands over information card and paper and pencil to both Candidates
You have 1 minute to decide together who’s going to choose which picture. So, each of you should choose one picture. If you want you can take notes and you may start when you are ready.
Then, you will have 4 minutes to talk together and persuade your partner/s about your choice, and explain why you haven’t chosen the other pictures.
Do you understand?
After 5 minutes retrieves information card
Thank you very much (Candidate 1) and (Candidate 2).
That is the end of the test.
1. Energy
‘Choose one of the following pictures and persuade your partner the energy source you have chosen is the best from an ecological and/or economic point of view.’
Photo credits: 1. stadtlohn-blog.de; 2. quabu.wordpress.com; 3. Claudio Pavesi
2. Means of transport
‘Choose one of these means of transport and persuade your partner the one you have chosen is the best to go on holiday to Paris.’
3. Friendship
‘Choose one of these ways of socialising and persuade your partner that it is the best one to stay in contact with your friends.’
4. Lifelong learning
‘Choose one of these methods of learning how to cook and persuade your partner that it is the most suitable.’
References
Bachman, Lyle F. 1990. Fundamental considerations in language testing. Oxford: Oxford University Press.Search in Google Scholar
Bachman, Lyle F. & Adrian Palmer. 1981. The construct validation of the FSI oral interview. Language Learning 31(1). 67–86.10.1111/j.1467-1770.1981.tb01373.xSearch in Google Scholar
Berry, Vivien. 2007. Personality differences and oral test performance. Frankfurt: Peter Lang.Search in Google Scholar
Brooks, Lindsay. 2003. Converting an observation checklist for use with the IELTS speaking test. Cambridge ESOL Research Notes 11. 277–303.Search in Google Scholar
Brooks, Lindsay. 2009. Interacting in pairs in a test of oral proficiency: Co-constructing a better performance. Language Testing 26(3). 341–366.10.1177/0265532209104666Search in Google Scholar
Brown, Annie. 2003. Interviewer variation and the co-construction of speaking proficiency. Language Testing 20(1). 277–303.10.1191/0265532203lt242oaSearch in Google Scholar
Brown, Annie. 2005. Interviewer variability in oral proficiency interviews. Frankfurt: Peter Lang.Search in Google Scholar
Bygate, Martin. 1988. Speaking. Oxford: Oxford University Press.Search in Google Scholar
Davies, Andrew. 1977. The construction of language tests. In J. P. B. Allen & Alan Davies (eds.), Testing and experimental methods. The Edinburgh course in applied linguistics, vol. 4, 38–194. Oxford: Oxford University Press.Search in Google Scholar
Davis, Larry. 2009. The influence of interlocutor proficiency in a paired oral assessment. Language Testing 26(3). 367–396.10.1177/0265532209104667Search in Google Scholar
Ffrench, Angela. 2003. The change process at the paper level. Paper 5, Speaking. In Cyril Weir & Michael Milanovic (eds.), Continuity and innovation: Revising the Cambridge Proficiency in English examination 1913–2002, 367–471. Cambridge: Cambridge University Press.Search in Google Scholar
Galaczi, Ewelina D. 2004. Peer-to-peer interaction in a paired speaking test: The case of the first certificate of English. New York: Columbia University Ph.D. thesis.Search in Google Scholar
Galaczi, Ewelina D. 2008. Peer-peer interaction in a speaking test: The case of the First Certificate in English examination. Language Assessment Quarterly 5(2). 89–119.10.1080/15434300801934702Search in Google Scholar
Henning, Grant. 1987. A guide to language testing. Cambridge, MA: Newbury House.Search in Google Scholar
Iwashita, Noriko. 1996. The validity of the paired interview format in oral performance assessment. Melbourne Papers in Language Testing 5(2). 51–66.Search in Google Scholar
Lazaraton, Anne. 2002. A qualitative approach to the validation of oral language tests. Cambridge: Cambridge University Press.Search in Google Scholar
Lazaraton, Anne. 2014. Spoken discourse. In Anthony J. Kunnan (ed.), The companion to language assessment, 1375–1389. Chichester: Wiley Blackwell.10.1002/9781118411360.wbcla023Search in Google Scholar
Lazaraton, Anne & Larry Davis. 2008. A microanalytic perspective on discourse proficiency and identity in paired oral assessment. Language Assessment Quarterly 5(4). 313–335.10.1080/15434300802457513Search in Google Scholar
Luoma, Suomi. 2004. Assessing speaking. Cambridge: Cambridge University Press.10.1017/CBO9780511733017Search in Google Scholar
May, Lynn. 2009. Co-constructed interaction in a paired speaking test: The rater’s perspective. Language Testing 26(3). 397–421.10.1177/0265532209104668Search in Google Scholar
Messick, Samuel. 1989. Validity. In Robert L. Linn (ed.), Educational measurement, 3rd edn. New York: Macmillan.Search in Google Scholar
Nakatsuhara, Fumiyo. 2006. The impact of proficiency level on conversational styles in paired speaking tests. Cambridge-ESOL Research Notes 25. 15–20.Search in Google Scholar
Nakatsuhara, Fumiyo. 2011. Effects of test-taker characteristics and the number of participants in group oral tests. Language Testing 28(4). 483–508.10.1177/0265532211398110Search in Google Scholar
Nakatsuhara, Fumiyo. 2013. The Co-construction of conversation in group oral tests. Frankfurt am Main: Peter Lang.10.3726/978-3-653-03584-1Search in Google Scholar
Nitta, Ryo & Fumiyo Nakatsuhara. 2014. A multifaceted approach to investigating pre-task planning effects on paired oral test performance. Language Testing 31(2). 147–175.10.1177/0265532213514401Search in Google Scholar
O’Sullivan, Barry. 2002. Learner acquaintanceship and oral proficiency test pair-task performance. Language Testing 19(3). 277–295.10.1191/0265532202lt205oaSearch in Google Scholar
O’Sullivan, Barry, Cyril Weir & Nick Saville. 2002. Using observation checklists to validate speaking-tests tasks. Language Testing 19(1). 33–56.10.1191/0265532202lt219oaSearch in Google Scholar
Ross, Steven & R. Berwick. 1992. The discourse of accommodation in oral proficiency interviews. Studies in Second Language Acquisition 14(2). 159–176.10.1017/S0272263100010809Search in Google Scholar
Sacks, Harvey, Emmanuel Schegloff & Gail Jefferson. 1974. A simplest systematics for the organisation of turn-taking for conversation. Language 50(4). 697–735.10.1353/lan.1974.0010Search in Google Scholar
Schegloff, Emmanuel, Gail Jefferson & Harvey Sacks. 1977. The preference for self-correction in the organization of repair in conversation. Language 53(2). 361–382.10.1353/lan.1977.0041Search in Google Scholar
Taylor, Linda. 2001. The paired speaking test format: Recent studies. Cambridge ESOL Research Notes 6. 15–17.Search in Google Scholar
Taylor, Linda. 2011. Examining speaking: Research and practice in assessing second language speaking. Cambridge: Cambridge University Press.Search in Google Scholar
Weir, Cyril J. 1993. Understanding and developing language tests. Hemel Hempstead: Prentice Hall.Search in Google Scholar
Weir, Cyril J. 2005. Language testing and validation: An evidence-based approach. Basingstoke: Palgrave MacMillan.10.1057/9780230514577Search in Google Scholar
Wong, Jean & Hansun Z. Waring. 2010. Conversation analysis and second language pedagogy: A guide for ESL/EFL teachers. New York: Routledge.10.4324/9780203852347Search in Google Scholar
Young, Richard & Michael Milanovich. 1992. Discourse variation in oral proficiency interviews. Studies in Second Language Acquisition 14(4). 403–424.10.1017/S0272263100011207Search in Google Scholar
© 2019 Walter de Gruyter GmbH, Berlin/Boston
