Abstract
This study investigates whether there is a relation between the semantics of linguistic expressions that indicate temporal distance and the spatial properties of their co-speech gestures. To this date, research on time gestures has focused on features such as gesture axis, direction, and shape. Here we focus on a gesture property that has been overlooked so far: the distance of the gesture in relation to the body. To achieve this, we investigate two types of temporal linguistic expressions are addressed: proximal (e.g., near future, near past) and distal (e.g., distant past, distant future). Data was obtained through the NewsScape library, a multimodal corpus of television news. A total of 121 co-speech gestures were collected and divided into the two categories. The gestures were later annotated in terms of gesture space and classified in three categories: (i) center, (ii) periphery, and (iii) extreme periphery. Our results suggest that gesture and language are coherent in the expression of temporal distance: when speakers locate an event far from them, they tend to gesture further from their body; similarly, when locating an event close to them, they gesture closer to their body. These results thus reveal how co-speech gestures also reflect a space-time mapping in the dimension of distance.
Research funding: This work was supported by the Spanish Ministerio de Ciencia, Innovación y Universidades, Agencia Estatal de Investigación & FEDER/UE funds (grant number PGC2018-1551 097658- B-100).
References
Beattie, Geoffrey & Heather Shovelton. 2011. An exploration of the other side of semantic communication: How the spontaneous movements of the human hand add crucial meaning to narrative. Semiotica 184. 33–51.10.1515/semi.2011.021Search in Google Scholar
Beattie, Geoffrey. 2016. Rethinking body language. London: Routledge.10.4324/9781315880181Search in Google Scholar
Bender, Andrea & Sieghard Beller. 2014. Mapping spatial frames of reference onto time: A review of theoretical accounts and empirical findings. Cognition 132(3). 342–382. https://doi.org/10.1016/j.cognition.2014.03.016.Search in Google Scholar
Boroditsky, Lera. 2000. Metaphoric structuring: Understanding time through spatial metaphors. Cognition 75. 1–28. https://doi.org/10.1016/s0010-0277(99)00073-6.Search in Google Scholar
Boroditsky, Lera. 2001. Does language shape thought?: English and Mandarin speaker’s conceptions of time. Cognitive Psychology 43. 1–22. https://doi.org/10.1006/cogp.2001.0748.Search in Google Scholar
Boroditsky, Lera & Michael Ramscar. 2002. The roles of body and mind in abstract thought. Psychological Science 13(2). 185–189. https://doi.org/10.1111/1467-9280.00434.Search in Google Scholar
Bressem, Jana & Silva Ladewig. 2011. Rethinking gesture phases: Articulatory features of gestural movement? Semiotica 184(1/4). 53–91.10.1515/semi.2011.022Search in Google Scholar
Bressem, Jana, Silva Ladewig & Cornelia Müller. 2013. Linguistic annotation system for gestures (LASG). In Cornelia Müller, Alan Cienki, Ellen Fricke, Silva Ladewig, David McNeill & Jana Bressem (eds.), Body – language – communication: An international handbook on multimodality in human interaction, 2nd ed. vol. 38, 1098–1124. Berlin: De Gruyter Mouton.10.1515/9783110261318.1098Search in Google Scholar
Bylund, Emanuel, Pascal Gygax, Steven Samuel & Panos Athanasopoulos. 2019. Back to the future? The role of temporal focus for mapping time onto space. Quarterly Journal of Experimental Psychology 73(2). 174–182.10.1177/1747021819867624Search in Google Scholar
Cai, Zhenguang G., Louise Connell & Judith Holler. 2013. Time does not flow without language: Spatial distance affects temporal duration regardless of movement or direction. Psychonomic Bulletin & Review 20. 973–980. https://doi.org/10.3758/s13423-013-0414-3.Search in Google Scholar
Calbris, Geneviève. 2008. From left to right … Coverbal gestures and their symbolic use of space. In Cornelia Müller & Alan Cienki (eds.), Metaphor and gesture, 27–55. Amsterdam & Philadelphia: John Benjamin.10.1075/gs.3.05calSearch in Google Scholar
Casasanto, Daniel & Lera Boroditsky. 2008. Time in the mind: Using space to think about time. Cognition 106. 579–593. https://doi.org/10.1016/j.cognition.2007.03.004.Search in Google Scholar
Casasanto, Daniel & Kyle Jasmin. 2012. The hands of time: Temporal gestures in English speakers. Cognitive Linguistics 23(4). 643–674. https://doi.org/10.1515/cog-2012-0020.Search in Google Scholar
Cienki, Alan & Cornelia Müller. 2008. Metaphor and gesture. Amsterdam, Philadelphia: John Benjamins Pub. Co.10.1075/gs.3Search in Google Scholar
Cienki, Alan, Aliyah Morgenstern, Cornelia Müller & Dominique Boutet. 2016. Linguistic aspect, tense and gestural movement quality in French, German, and Russian utterances, vol. 7. Paris: International Society of Gestures Studies.Search in Google Scholar
Cienki, Alan. 1998. Metaphoric gestures and some of their relations to verbal metaphoric expressions. In Jean-Pierre Koenig (ed.), Discourse and cognition: Bridging the gap, 189–204. Stanford, CA: CSLI Publications.Search in Google Scholar
Cienki, Alan. 2008. Why study metaphor and gesture? In Alan Cienki & Cornelia Müller (eds.), Metaphor and gesture, 5–25. Cambridge: Cambridge University Press. https://doi.org/10.1075/gs.3.04cie.Search in Google Scholar
Cienki, Alan. 2013. Cognitive linguistics: Language and gestures as expressions of conceptualisation. In Cornelia Müller, Alan Cienki, Ellen Fricke, Silva Ladewig, David McNeill & Sedinha Tessendorf (eds.), Body-language-communication: An international handbook on multimodality in human interaction, 182–201. Berlin: De Gruyter Mouton.Search in Google Scholar
Cooperrider, Kensy & Rafael Núñez. 2009. Across time, across the body: Transversal temporal gestures. Gesture 9(2). 181–206. https://doi.org/10.1075/gest.9.2.02coo.Search in Google Scholar
Cohen, Doron, Geoffrey Beattie & Heather Shovelton. 2010. Nonverbal indicators of deception: How iconic gestures reveal thoughts that cannot be suppressed. Semiotica 182. 133–174.10.1515/semi.2010.055Search in Google Scholar
Colapietro, Vincent. 2001. A lantern for the feet of inquirers: The heuristic function of the Peircean categories. Semiotica 136(1/4). 201–216.10.1515/semi.2001.074Search in Google Scholar
Coventry, Kenny R., Berenice Valdés, Alejandro Castillo & Pedro Guijarro-Fuentes. 2008. Language within your reach: Near–far perceptual space and spatial demonstratives. Cognition 108. 889–895. https://doi.org/10.1016/j.cognition.2008.06.010.Search in Google Scholar
Davis, Stephen Boyd. 2012. History on the line: Time as dimension. Design Issues 26(4). 4–17. https://doi.org/10.1162/desi_a_00171.Search in Google Scholar
Danaher, David. 1998. Peirce’s semiotic and cognitive metaphor theory. Semiotica 119(1/2). 171–207.Search in Google Scholar
de la Fuente, Juanma, Julio Santiago, Antonio Román, Cristina Dumitrache & Daniel Casasanto. 2014. When you think about it, your past is in front of you: How culture shapes spatial conceptions of time. Psychological Science 29. 1682–1690. https://doi.org/10.1177/0956797614534695.Search in Google Scholar
Diessel, Holger. 1999. Demonstratives: Form, function, and grammaticalization. Amsterdam/Philadelphia: John Benjamins.10.1075/tsl.42Search in Google Scholar
Diessel, Holger. 2006. Demonstratives, joint attention, and the emergence of grammar. Cognitive Linguistics 17(4). 463–489. https://doi.org/10.1515/cog.2006.015.Search in Google Scholar
Diessel, Holger. 2013. Distance contrasts in demonstratives. In Matthew S. Dryer & Martin Haspelmath (eds.), The world atlas of language structures online. Leipzig: Max Planck Institute for Evolutionary Anthropology.Search in Google Scholar
Duffy, Sarah. 2014. The role of cultural artifacts in the interpretation of metaphorical expressions about time. Metaphor and Symbol 29(2). 94–112. https://doi.org/10.1080/10926488.2014.889989.Search in Google Scholar
Farias, Priscila & Joao Queiroz. 2006. Images, diagrams, and metaphors: Hypoicons in the context of Peirce’s sixty-six-fold classification of signs. Semiotica 162(1/4). 287–307.10.1515/SEM.2006.081Search in Google Scholar
Fauconnier, Gilles & Mark Turner. 2008. Rethinking metaphor. New York: Cambridge University Press.10.1017/CBO9780511816802.005Search in Google Scholar
Goldin-Meadow, Susan. 2003. Hearing gesture: How our hands help us think. Cambridge: Belknap Press of Harvard University Press.10.1037/e413812005-377Search in Google Scholar
Goldin-Meadow, Susan. 2004. Gestures role in the learning process. Theory Into Practice 43. 314–321. https://doi.org/10.1207/s15430421tip4304_10.Search in Google Scholar
Gullberg, Marianne & Sotaro Kita. 2009. Attention to speech-accompanying gestures: Eye movements and information uptake. Journal of Nonverbal Behavior 33. 251–277. https://doi.org/10.1007/s10919-009-0073-2.Search in Google Scholar
Gullberg, Marianne & Kenneth Holmqvist. 1999. Keeping an eye on gestures: Visual perception of gestures in face-to-face communication. Pragmatics and Cognition 7. 35–63. https://doi.org/10.1075/pc.7.1.04gul.Search in Google Scholar
Gullberg, Marianne & Kenneth Holmqvist. 2002. Visual attention towards gestures in face-to-face interaction vs. on screen. In Ipke Wachsmuth & Timo Sowa (eds.), Gesture and sign language in human-computer interaction. GW 2001. Berlin, Heidelberg: Springer.10.1007/3-540-47873-6_23Search in Google Scholar
Gullberg, Marianne & Kenneth Holmqvist. 2006. What speakers do and what listeners look at: Visual attention to gestures in human interaction live and on video. Pragmatics and Cognition 14. 53–82. https://doi.org/10.1075/pc.14.1.05gul.Search in Google Scholar
Iriskhanova, Olga & Alan Cienki. 2018. The semiotics of gestures in cognitive linguistics: Contribution and challenges. Voprosy Kognitivnoy Lingvistiki 4. 25–36. https://doi.org/10.20916/1812-3228-2018-4-25-36.Search in Google Scholar
Iverson, Jana & Susan Goldin-Meadow. 1998. Why people gesture as they speak. Nature 396. 228. https://doi.org/10.1038/24300.Search in Google Scholar
Kemmerer, David. 1999. “Near” and “far” in language and perception. Cognition 73(1). 35–63. https://doi.org/10.1016/s0010-0277(99)00040-2.Search in Google Scholar
Kendon, Adam. 1980. Gesture and speech: Two aspects of the process of utterance. In Mary R. Key (ed.), Nonverbal communication and language, 207–227. The Hague: Mouton.10.1515/9783110813098.207Search in Google Scholar
Kendon, Adam. 2000. Language and gesture: Unity or duality. In David McNeill (ed.), Language and gesture: Window into thought and action, 47–63. Cambridge: Cambridge University Press.10.1017/CBO9780511620850.004Search in Google Scholar
Kendon, Adam. 2004. Gesture: Visible action as utterance. Cambridge: Cambridge University Press.10.1017/CBO9780511807572Search in Google Scholar
Kranjec, Alexander & Anjan Chatterjee. 2010. Are temporal concepts embodied? A challenge for cognitive neuroscience. Frontiers in Psychology 1. 240. https://doi.org/10.3389/fpsyg.2010.00240.Search in Google Scholar
Kranjec, Alexander & Laraine McDonough. 2011. The implicit and explicit embodiment of time. Journal of Pragmatics 43(3). 735–748. https://doi.org/10.1016/j.pragma.2010.07.004.Search in Google Scholar
Lakoff, George & Mark Johson. 1980. Metaphors we live by. Chicago: Chicago University Press.Search in Google Scholar
Lakoff, George. 1987. Women, fire, and dangerous things: what categories reveal about the mind. Chicago: University of Chicago Press.10.7208/chicago/9780226471013.001.0001Search in Google Scholar
Lapaire, Jean-Remi. 2016. From ontological metaphor to semiotic make-believe: Giving shape and substance to fictive objects of conception with the globe gesture. Signo 41. 70. https://doi.org/10.17058/signo.v41i70.6413.Search in Google Scholar
Levinson, Stephen & Asifa Majid. 2013. The island of time: Yélî Dnye, the language of Rossel island. Frontiers in Psychology 4. 61. https://doi.org/10.3389/fpsyg.2013.00061.Search in Google Scholar
Matlock, Teenie, Michael Ramscar & Lera Boroditsky. 2005. On the experiential link between spatial and temporal language. Cognitive Science 29(4). 655–664. https://doi.org/10.1207/s15516709cog0000_17.Search in Google Scholar
McNeill, David. 1985. So you think gestures are nonverbal? Psychological Review 92. 350–371. https://doi.org/10.1037/0033-295x.92.3.350.Search in Google Scholar
McNeill, David. 1992. Hand and mind. Chicago: University of Chicago Press.Search in Google Scholar
McNeill, David. 2005. Gesture and thought. Chicago: University of Chicago Press.10.7208/chicago/9780226514642.001.0001Search in Google Scholar
Mittelberg, Irene. 2019. Peirce’s universal categories: On their potential for gesture theory and multimodal analysis. Semiotica 228. 193–222. https://doi.org/10.1515/sem-2018-0090.Search in Google Scholar
Mittelberg, Irene. 2008. Peircean semiotics meets conceptual metaphor: Iconic modes in gestural representations of grammar. In Alan Cienki & Cornelia Müller (eds.), Metaphor and gesture, 115–154. Amsterdam & Philadelphia: John Benjamins.10.1075/gs.3.08mitSearch in Google Scholar
Moore, Kevin. 2006. Space-to-time mappings and temporal concepts. Cognitive Linguistics 17. 199–244. https://doi.org/10.1515/cog.2006.005.Search in Google Scholar
Moore, Kevin. 2011. Ego-Perspective and field-based frames of reference: Temporal meanings of FRONT in Japanese, Wolof and Aymara. Journal of Pragmatics 43(3). 759–776. https://doi.org/10.1016/j.pragma.2010.07.003.Search in Google Scholar
Ng, Melvin R., Winston D. Goh, Melvin J. Yap, Chi-Sing Tse & Wing-Chee So. 2017. We think about temporal words: A gestural priming study in English and Chinese. Friontiers in Psychology 8. 974. https://doi.org/10.3389/fpsyg.2017.00974.Search in Google Scholar
Nöth, Winfried. 1990. Handbook of semiotics. Bloomington: Indiana University Press.10.2307/j.ctv14npk46Search in Google Scholar
Nöth, Winfried. 1999. Peircean semiotics in the study of iconicity in language. Transactions of the Charles S. Peirce Society 35(3). 613–619.Search in Google Scholar
Núñez, Rafael & Eve Sweetser. 2006. With the future behind them: Converge evidence from Aymara language and gesture in the crosslinguistic comparison of spatial construals of time. Cognitive Science 30. 1–49.10.1207/s15516709cog0000_62Search in Google Scholar
Özyrükek, Asli. 2014. Hearing and seeing meaning in speech and gesture: Insights from brain and behaviour. Philosophical Transactions of the Royal Society of London Series B Biological Sciences 369. 1651.10.1098/rstb.2013.0296Search in Google Scholar
Pagán Cánovas, Cristóbal, Javier Valenzuela, Alcaraz Carrión Daniel, Olza Inés & Ramscar Michael. 2020. Quantifying the speech-gesture relation with massive multimodal datasets: Informativity in time expressions. PloS One 15(6). e0233892. https://doi.org/10.1371/journal.pone.0233892.Search in Google Scholar
Peirce, Charles S. 1931–1966. The collected papers of Charles S. Peirce. In Charles Hartshorne, Paul Weiss & Arthur W. Burks (eds.), vol. 8. Cambridge: Harvard University Press [Reference to Peirce’s papers will be designated CP followed by volume and paragraph number].Search in Google Scholar
Santiago, Julio, Juan Lupiáñez, Elvira Pérez & Maria José Funes. 2007. Time also flies from left to right. Psychonomic Bulletin & Review 14(3). 512–516. https://doi.org/10.3758/bf03194099.Search in Google Scholar
Torralbo, Ana, Julio Santiago & Juan Lupiañez. 2006. Flexible conceptual projection of time onto spatial frames of reference. Cognitive Science 30. 745–757. https://doi.org/10.1207/s15516709cog0000_67.Search in Google Scholar
Valenzuela, Javier, Cristobal Pagán-Cánovas, Inés Olza & Daniel Alcaraz. 2020. Gesturing in the wild: Spontaneous gestures co-occurring with temporal demarcative expressions provide evidence for a flexible mental timeline. Review on Cognitive Linguistics 18(2). 289–316. https://doi.org/10.1075/rcl.00061.val.Search in Google Scholar
Walker, Esther & Kensy Cooperrider. 2016. The continuity of metaphor: Evidence from temporal gestures. Cognitive Science 40(2). 481–495. https://doi.org/10.1111/cogs.12254.Search in Google Scholar
© 2021 Walter de Gruyter GmbH, Berlin/Boston