Iconicity in speech and gesture

Organizer(s) Susanne Fuchs, Cornelia Ebert, Aleksandra Ćwiek and Manfred Krifka
Start of event 14.05.2018, 09.00 o'clock
End of event 15.05.2018, 16.00 o'clock
Venue ZAS Berlin
Program & Abstracts

Invited Speakers:

Mark Dingemanse
Stefan Kopp
Markus Steinbach
Bodo Winter


Monday, May 14th


Welcome (Manfred Krifka)


Markus Steinbach: Constructing Meaning at the Interface between Sign and Gesture


Coffee break


Mark Dingemanse: Making and breaking iconicity


Lunch break


Bodo Winter: Where is iconicity and why isn't there more of it?


Lab tour


Story telling performance “Erzählbühne” 
ZENTRUM danziger50 
Danziger Str. 50, 10435 Berlin 

Tuesday, May 15th


Cornelia Ebert: Experimental studies on the semantic contribution of co- and post-speech gestures and their significance for perspective-taking


Coffee break


Round table “Show & Tell” 


Lunch break


Stefan Kopp: Embedded iconicity — what computational studies reveal about co-speech iconic gestures


Aleksandra Ćwiek: A tiny tiny abstract on a biiiiiig topic – An analysis of adjective re(du)plications in a social media corpus


Aleksandra Ćwiek
A tiny tiny abstract on a biiiiiig topic – An analysis of adjective re(du)plications in a social media corpus
(joint work with Susanne Fuchs, Egor Savin, Cornelia Ebert & Manfred Krifka)

In the absence of face-to-face communication speakers use their linguistic creativity in developing new strategies to make up for the absence of other means to transfer information. Social media corpora offer an excellent possibility to study such novelties, as users of social media platforms do not necessarily follow standard orthographic and syntactic rules. To compensate the lack of prosodic domain and audiovisual information a new way of intensifying words in written text is applied: (1) letter replication, as in “loooooooonnnnngggg” (Brody & Diakopoulos, 2011), or (2) word reduplication, as in big big world.

In this talk, we will present a corpus study analyzing the frequency of letter replications in antonym pairs of gradable size and word reduplications of the same pairs. The analysis was conducted on a freely available English social media corpus (Schler et al., 2006) including ca. 140 million words from 19000 bloggers. We will present evidence for iconic use of orthographic means, compensating for prosodic iconicity in the absence of spoken communication channel.

Mark Dingemanse
Making and breaking iconicity

Research on iconicity has long focused on the iconic associations that people make – from cross-modal mappings in pseudowords like bouba and kiki to perceptual analogies in ideophones and gestures that help learning and communication. For a more fundamental understanding, we also need to study what happens when we break iconicity. When you break a system (carefully, in a controlled way), you can get a new view of its internals. In this talk I discuss iconicity from both perspectives. I explore what shapes and constrains iconicity in the lexicon and in language use, combining cross-linguistic experimental work with corpus-based approaches. Using corpus data from Japanese and Siwu (a Kwa language spoken in Ghana), I show when ideophones are at their most iconic, and under which circumstances they are more likely to be treated as ordinary words. Using data from learning and guessing tasks with 240 ideophones from 5 languages, I show how iconicity makes ideophones easier to learn, but only when phonological and prosodic mappings between form and meaning are preserved. If we want to understand the cognitive processes and pragmatic principles underlying iconicity, breaking iconicity is as important methodologically as making it.

Cornelia Ebert
Experimental studies on the semantic contribution of co- and post-speech gestures and their significance for perspective-taking
(in parts joint work with Aleksandra Ćwiek, Susanne Fuchs & Manfred Krifka)

In this talk, we will present different studies and experimental work in progress on the interplay of gesture and speech. 
Recently, two different formal approaches have been put forth to explain the semantic behaviour of co-speech gestures (gestures that accompany speech) and post-speech gestures (gestures that follow speech). Both approaches argue that, by default, gesture meaning enters into composition as non-at-issue material. They differ in the proposed nature of the non-at-issueness, which is supposition-like for Ebert & Ebert (2014) or co-suppositional, i.e. a special kind of presupposition, for Schlenker (to appear). 
We will first present experimental evidence for the claim that gestures are non-at-issue and look at experimental options to test for this hypothesis. Then we will contrast the different predictions of Ebert & Ebert as opposed to those of Schlenker and present preliminary results of a pilot study that tests for these predictions. Finally, we will look at two studies that aim at testing for perspective taking and viewpoint shift via manipulation of overt viewpoint gestures. One study investigates context shift in free indirect discourse, the other possibilities of perspective change within a sentence.

Stefan Kopp
Embedded iconicity — what computational studies reveal about co-speech iconic gestures

In this talk, I will present several approaches to study iconic co-speech gesture using computational methods. This will include (1) analyses of crowd-sourced rating data to investigate the multi-functionality of gesture, (2) perceptual experiments with synthetic gesture generate with embodied artificial agents, and (3) computations models of the production of speech and gesture to unravel the underlying cognitive processes and mechanisms.

Markus Steinbach
Constructing Meaning at the Interface between Sign and Gesture

Sign language and gesture share the same visual-gestural modality. Therefore, signers – like speakers – cannot only use co-speech gestures accompanying utterances but also – unlike speakers – systematically integrate manual and non-manual gestures in various ways into utterances. A particularly interesting example for the systematic interaction of gesture and sign language is constructed action (i.e. action role shift), which has been the focus of much debate in recent literature on meaning and modality. In this presentation, I'll discuss various examples of gestural meaning components that are integrated into the proposition expressed by the utterance and show how recent accounts of action role shift (Cormier et al. 2015, Davidson 2015, Maier 2015, Herrmann/Pendzich 2018, Schlenker to appear-a,b) can explain the modality-specific interaction of gesture and sign.

Bodo Winter
Where is iconicity and why isn’t there more of it?

More and more studies show that spoken and signed languages harbor a considerable degree of iconicity, form-meaning mappings that are motivated by resemblance, such as the word “bang”, which sounds somewhat like an actual bang. In this talk, I will consider iconicity as a graded quantity, with certain words being more or less iconic. Rather than asking the question, “Are languages iconic or arbitrary?”, I will be asking, “Where is iconicity?” Using English iconicity scores from a rating study, I will show that iconicity is ramped up in children’s speech and the child-directed speech of adults. I will then show that iconicity is also heightened in perceptual language, in particular for sound words (“squealing”, “beeping”, “rustling”) and touch words (“rough”, “smooth”, “prickly”). These analyses of the distribution of iconicity within language also help us understand why languages are characterized by a lot of arbitrariness. In particular, I will argue that it is precisely the connection between iconicity and perceptual semantics which restricts its domain of use. To this end, I will report new quantitative evidence which shows that iconicity is inimical to abstraction in language.