Hostname: page-component-8448b6f56d-qsmjn Total loading time: 0 Render date: 2024-04-23T07:22:49.879Z Has data issue: false hasContentIssue false
  • English
  • Français

Developing, Analyzing and Sharing Multivariate Datasets: Individual Differences in L2 Learning Revisited

Published online by Cambridge University Press:  30 June 2020

Kazuya Saito ,
Konstantinos Macmillan ,
Tran Mai ,
Yui Suzukida ,
Hui Sun ,
Viktoria Magne ,
Meltem Ilkan  and
Akira Murakami
Kazuya Saito*
Affiliation:
University College London, UK
Konstantinos Macmillan
Affiliation:
University of London, UK
Tran Mai
Affiliation:
University of London, UK
Yui Suzukida
Affiliation:
University College London, UK
Hui Sun
Affiliation:
University of London, UK
Viktoria Magne
Affiliation:
University of West London, UK
Meltem Ilkan
Affiliation:
University of London, UK
Akira Murakami
Affiliation:
University of Birmingham, UK
*
*Corresponding author. E-mail: k.saito@ucl.ac.uk
Get access Rights & Permissions [Opens in a new window]

Abstract

Following the trends established in psychology and emerging in L2 research, we explain our support for an Open Science approach in this paper (i.e., developing, analyzing and sharing datasets) as a way to answer controversial and complex questions in applied linguistics. We illustrate this with a focus on a frequently debated question, what underlies individual differences in the dynamic system of post-pubertal L2 speech learning? We provide a detailed description of our dataset which consists of spontaneous speech samples, elicited from 110 late L2 speakers in the UK with diverse linguistic, experiential and sociopsychological backgrounds, rated by ten L1 English listeners for comprehensibility and nativelikeness. We explain how we examined the source of individual differences by linking different levels of L2 speech performance to a range of learner-extrinsic and intrinsic variables related to first language backgrounds, age, experience, motivation, awareness, and attitudes using a series of factor and Bayesian mixed-effects ordinal regression analyses. We conclude with a range of suggestions for the fields of applied linguistics and SLA, including the use of Bayesian methods in analyzing multivariate, multifactorial data of this kind, and advocating for publicly available datasets. In keeping with recommendations for increasing openness of the field, we invite readers to rethink and redo our analyses and interpretations from multiple angles by making our dataset and coding publicly available as part of our 40th anniversary ARAL article.

Type
Research Article
Information
Annual Review of Applied Linguistics , Volume 40 , March 2020 , pp. 9 - 25
Check for updates
Open Practices
Open materials Open data
Copyright
Copyright © The Author(s), 2020. Published by Cambridge University Press

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

The experiment in this article earned an Open Data and an Open Materials badge for transparent practices. The data and materials are available at https://www.iris-database.org/iris/app/home/detail?id=york%3a937998&ref=search (Dataset, Scores, & Variables); https://www.iris-database.org/iris/app/home/detail?id=york%3a937999&ref=search (RCode & URL); and https://www.iris-database.org/iris/app/home/detail?id=york%3a937997&ref=search (Supporting Information).

References

Best, C., & Tyler, M. (2007). Nonnative and second-language speech perception. In Bohn, O. & Munro, M. (Eds.), Language experience in second language speech learning: In honour of James Emil Flege (pp. 13–34). John Benjamins.CrossRefGoogle Scholar
Boersma, P., & Weenink, D. (2017). PRAAT: doing phonetics by computer [Computer program]. Version 6.0.28.Google Scholar
Bongaerts, T., van Summeren, C., Planken, B., & Schils, E. (1997). Age and ultimate attainment in the pronunciation of a foreign language. Studies in Second Language Acquisition, 19(4), 447–465.CrossRefGoogle Scholar
Bürkner, P. (2017). brms: An R package for Bayesian multilevel models using stan. Journal of Statistical Software, 80, 1–28.CrossRefGoogle Scholar
Bürkner, P. C., & Vuorre, M. (2019). Ordinal regression models in psychology: A tutorial. Advances in Methods and Practices in Psychological Science, 2(1), 77–101.CrossRefGoogle Scholar
Carpenter, B., Gelman, A., Hoffman, M. D., Lee, D., Goodrich, B., Betancourt, M., Brubaker, M., Guo, J., Li, P., & Riddell, A. (2017). Stan: A probabilistic programming language. Journal of Statistical Software, 76(1), 1–32. doi: 10.18637/jss.v076.i01CrossRefGoogle Scholar
Derwing, T. M., & Munro, M. J. (2013). The development of L2 oral language skills in two L1 groups: A 7-year study. Language Learning, 63(2), 163–185.CrossRefGoogle Scholar
Dewaele, J. M., Witney, J., Saito, K. & Dewaele, L. (2018). Foreign language enjoyment and anxiety in the FL classroom: The effect of teacher and learner variables. Language Teaching Research, 22(6), 676–697. doi: 10.1177/1362168817692161CrossRefGoogle Scholar
Elliott, A. R. (1995). Field independence/dependence, hemispheric specialization, and attitude in relation to pronunciation accuracy in Spanish as a foreign language. The Modern Language Journal, 79(3), 356–371.CrossRefGoogle Scholar
Flege, J. E., Munro, M. J., & MacKay, I. R. (1995). Factors affecting strength of perceived foreign accent in a second language. The Journal of the Acoustical Society of America, 97(5), 3125–3134.CrossRefGoogle Scholar
Flege, J. E., MacKay, I. R., & Piske, T. (2002). Assessing bilingual dominance. Applied Psycholinguistics, 23(4), 567–598.CrossRefGoogle Scholar
Freed, B., Dewey, D., Segalowitz, N., & Halter, R. (2004). The language contact profile. Studies in Second Language Acquisition, 26(2), 349–356.CrossRefGoogle Scholar
Gardner, R.C., & Smythe, P.C. (1981). On the development of the Attitude/Motivation Test Battery. Canadian Modern Language Review, 37(3), 510–525.CrossRefGoogle Scholar
Garrett, P. (2010). Attitudes to language. Cambridge University Press.CrossRefGoogle Scholar
Gatbonton, E., & Trofimovich, P. (2008). The ethnic group affiliation and L2 proficiency link: Empirical evidence. Language Awareness, 17(3), 229–248.CrossRefGoogle Scholar
Gerrig, R., & Rastle, K. (2019). New initiatives to promote open science at the Journal of Memory and Language. Journal of Memory and Language, 104, 126–127.CrossRefGoogle Scholar
Gewin, V. (2016). Data sharing: An open mind on open data. Nature, 529, 117–119.CrossRefGoogle ScholarPubMed
Hama, M., & Leow, R. P. (2010). Learning without awareness revisited: Extending Williams (2005). Studies in Second Language Acquisition, 32(3), 465–491.CrossRefGoogle Scholar
Harrell, F. E. Jr. (2015). Regression modeling strategies: With applications to linear models, logistic regression, and survival analysis (2nd edition). Springer.CrossRefGoogle Scholar
Hopp, H., & Schmid, M. S. (2013). Perceived foreign accent in first language attrition and second language acquisition: The impact of age of acquisition and bilingualism. Applied Psycholinguistics, 34(2), 361–394.CrossRefGoogle Scholar
Hui, B. (2019). Analyzing processing time data in applied linguistics and second language research: A multivariate mixed-effects approach. Journal of Research Design and Statistics in Linguistics and Communication Science, 5(1–2), 189–207. https://doi.org/10.1558/jrds.39117CrossRefGoogle Scholar
Ioup, G., Boustagi, E., El Tigi, M., & Moselle, M. (1994). Reexamining the critical period hypothesis: A case study of successful adult SLA in a naturalistic environment. Studies in Second Language Acquisition, 16(1), 73–98.CrossRefGoogle Scholar
Jia, G., & Aaronson, D. (2003). A longitudinal study of Chinese children and adolescents learning English in the United States. Applied Psycholinguistics, 24(1), 131–161.CrossRefGoogle Scholar
Kruschke, J. K. (2014). Doing Bayesian data analysis: A tutorial with R, JAGS, and Stan (2nd edition). Academic Press.Google Scholar
Lambert, B. (2018). A student's guide to Bayesian statistics. Sage Publications.Google Scholar
Larsen-Freeman, D. (2012). Complex, dynamic systems: A new transdisciplinary theme for applied linguistics? Language Teaching, 45(2), 202–214.CrossRefGoogle Scholar
Larson-Hall, J., & Plonsky, L. (2015). Reporting and interpreting quantitative research findings: What gets reported and recommendations for the field. Language Learning, 65(S1), 127–159.CrossRefGoogle Scholar
Liddell, T. M., & Kruschke, J. K. (2018). Analyzing ordinal data with metric models: What could possibly go wrong? Journal of Experimental Social Psychology, 79, 328–348.CrossRefGoogle Scholar
Loewen, S., & Gonulal, T. (2015). Exploratory factor analysis and principal components analysis. In Plonsky, L. (Ed.), Advancing quantitative methods in second language research (pp. 182–212). Routledge.CrossRefGoogle Scholar
Marsden, E. (in press). Open science in applied linguistics and its consequences for quality and scope of research. In McKinley, J. and Rose, H. (Eds.), Routledge handbook of research methods in applied linguistics. Routledge.Google Scholar
Marsden, E., Crossley, S., Ellis, N., Kormos, J., Morgan-Short, K., & Thierry, G. (2019). Inclusion of Research Materials When Submitting an Article to Language Learning. Language Learning, 69(4), 795–801.CrossRefGoogle Scholar
Marsden, E., Mackey, A., & Plonsky, L. (2016). The IRIS repository: Advancing research practice and methodology. In Mackey, A. & Marsden, E. (Eds.), Advancing methodology and practice: The IRIS repository of instruments for research into second languages (pp. 1–21). Routledge.Google Scholar
Marsden, E., Morgan-Short, K., Thompson, S. & Abugaber, D. (2018). Replication in second language research: narrative and systematic reviews and recommendations for the field. Language Learning, 68(2), 321–391.CrossRefGoogle Scholar
Marsden, E., Thompson, S., & Plonsky, L. (2018). A methodological synthesis of self-paced reading in second language research. Applied Psycholinguistics, 39(5), 861–904. doi:10.1017/S0142716418000036CrossRefGoogle Scholar
McKenzie, R. M., & Gilmore, A. (2017). “The people who are out of ‘right’ English”: Japanese university students’ social evaluations of English language diversity and the internationalisation of Japanese higher education. International Journal of Applied Linguistics, 27(1), 152–175.CrossRefGoogle Scholar
McKiernan, E. C., Bourne, P. E., Brown, C. T., Buck, S., Kenall, A., Lin, J., McDougall, D., Nosek, B. A., Ram, K., Soderberg, C. K., Spies, J. R., Thaney, K., Updegrove, A., Woo, K. H., & Yarkoni, T. (2016). How open science helps researchers succeed. eLife, 5, e16800.CrossRefGoogle ScholarPubMed
Muñoz, C. (2014). Contrasting effects of starting age and input on the oral performance of foreign language learners. Applied Linguistics, 35, 463–482.CrossRefGoogle Scholar
Muñoz, C., & Llanes, À. (2014). Study abroad and changes in degree of foreign accent in children and adults. The Modern Language Journal, 98(1), 432–449.CrossRefGoogle Scholar
Munro, M. (2013). What do you know when you “know” an L2 vowel? In Paper presented at Pronunciation in Second Language Learning and Teaching Conference, Ames, IA.Google Scholar
Norouzian, R., de Miranda, M., & Plonsky, L. (2018). The Bayesian revolution in second language research: An applied approach. Language Learning, 68(4), 1032–1075.CrossRefGoogle Scholar
Ortega, L. (2018). Ontologies of language, second language acquisition, and world Englishes. World Englishes, 37(1), 64–79.CrossRefGoogle Scholar
Piske, T., Flege, J., MacKay, , & Meador, D. (2011). Investigating native and non-native vowels produced in conversational speech. In Wrembel, M., Kul, M. & Dziubalska-Kołaczyk, K. (Eds.), Achievements and perspectives in the acquisition of second language speech: New Sounds 2010 (pp. 195–205). Peter Lang.Google Scholar
Plonsky, L., Marsden, E. J., Crowther, D., Gass, S., & Spinner, P. (2019). A methodological synthesis and meta-analysis of judgment tasks in second language research. Second Language Research, 1–39.Google Scholar
Saito, K. (2019). Individual differences in second language speech learning in classroom settings: Roles of awareness in the longitudinal development of Japanese learners’ English /ɹ/ pronunciation. Second Language Research, 35(2), 149–172.CrossRefGoogle Scholar
Saito, K., & Munro, M. (2014). The early phase of /r/ production development in adult Japanese learners of English. Language and Speech, 57(4), 451–469.CrossRefGoogle Scholar
Saito, K., & Plonsky, L. (2019). Effects of second language pronunciation teaching revisited: A proposed measurement framework and meta-analysis. Language Learning, 69(3), 652–708.CrossRefGoogle Scholar
Saito, K., Trofimovich, P., & Isaacs, T. (2016). Second language speech production: Investigating linguistic correlates of comprehensibility and accentedness for learners at different ability levels. Applied Psycholinguistics, 37(2), 217–240.CrossRefGoogle Scholar
Saito, K., Trofimovich, P., & Isaacs, T. (2017). Using listener judgments to investigate linguistic influences on L2 comprehensibility and accentedness: A validation and generalization study. Applied Linguistics, 38(4), 439–462.Google Scholar
Schmidt, R. (2001). Attention. In Robinson, P. (Ed.), Cognition and second language instruction (pp. 1–32). Cambridge University Press.Google Scholar
Silberzahn, R., Uhlmann, E. L., Martin, D. P., Anselmi, P., Aust, F., Awtrey, E., & Nosek, B. A. (2018). Many analysts, one data set: Making transparent how variations in analytic choices affect results. Advances in Methods and Practices in Psychological Science, 3, 337–356.CrossRefGoogle Scholar
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22(11), 1359–1366.CrossRefGoogle ScholarPubMed
Steegen, S., Tuerlinckx, F., Gelman, A., & Vanpaemel, W. (2016). Increasing transparency through a multiverse analysis. Perspectives on Psychological Science, 11(5), 702–712.CrossRefGoogle ScholarPubMed
Suzuki, S., & Kormos, J. (2019). Linguistic dimensions of comprehensibility and perceived fluency: An investigation of complexity, accuracy, and fluency in second language argumentative speech. Studies in Second Language Acquisition, 1–25. doi:10.1017/S0272263119000627Google Scholar
Venkatagiri, H. S., & Levis, J. M. (2007). Phonological awareness and speech comprehensibility: An exploratory study. Language Awareness, 16(4), 263–277.CrossRefGoogle Scholar
Supplementary material: File

Saito et al. supplementary material

Saito et al. supplementary material

Download Saito et al. supplementary material(File)
File 5.3 MB