An interdisciplinary research team has developed a workflow designed to improve research on language and communication by integrating nonverbal information. From ZAS, Dr. Susanne Fuchs, a researcher in Research Area 1 (Laboratory Phonology), and Dr. Aleksandra Ćwiek, a former researcher in Research Area 1 (Laboratory Phonology), were involved.
The scientists aim to make the investigation of language and communication more transparent while giving greater consideration to nonverbal information. Signals from different modalities—spoken language, manual and non-manual gestures, and other meaning-bearing elements—are closely intertwined and collectively contribute to the conveyance of meaning. The elicitation of such multimodal data is labor-intensive, costly, and methodologically challenging. In the publication, the researchers present a framework designed to guide the collection and analysis of multimodal data, while leaving room to tailor the research process to the specific research question.
The study was funded by the German Research Foundation (DFG) as part of the Priority Program “Visual Communication” (ViCom). The workflow was described in the study titled “Data Collection in Multimodal Language and Communication Research: A Flexible Decision Framework” in the journal “Advances in Methods in Psychology. Link to the paper: https://doi.org/10.1177/25152459261442338
For more details click the link below to a blog post on the priority program’s website. There you will also find a video featuring the translation of the blog post into sign language. https://vicom.info/new-publication-in-advances-in-methods-and-practices-in-psychological-science-by-vicom-researchers/