Leibniz-Zentrum Allgemeine Sprachwissenschaft Leibniz-Gemeinschaft


Motion capture laboratory

Whoever has seen a making-off of a modern Hollywood movie might be familiar with some elements of the motion capture lab at ZAS. The prerequisite is that the film contains characters that were animated on the computer using motion capturing.

The most eye-catching element is probably the small, silvery spheres, the "markers". The test subject carries these markers at movable points, mostly at joints of the body, but also in the face. The markers are either attached to a black special suit or to the body with double-sided adhesive tape. Twelve cameras are mounted just below the ceiling in the laboratory and illuminate an area of about 5 square meters with infrared light. If a marker is in the field of vision of at least two of the cameras, the system can clearly determine its position: The reflective surface of the markers appears particularly bright in the images of the cameras. All image information below a certain threshold value is discarded so that only a white point cloud on a black background is recorded. The three-dimensional position of the markers in space is calculated by triangulation using information about camera geometry and optics and the combination of images from several cameras. This is the same principle used by geologists for land surveying.

Data can be recorded at a rate of up to 220 frames per second. The result is long series of numbers (three spatial components per marker, making 3×37×220×60×5=~7Mio values for the recording of the entire skeleton over a period of five minutes for 37 markers) from which it is possible to extract further parameters such as distance travelled, speed or joint angle.

What do motion recordings have to do with linguistics at ZAS? When we reproduce our thoughts in the form of speech, we activate a variety of muscles, move the tongue and other articulators to make sounds, syllables, words. We also move our eyebrows (facial expressions) in certain moments, nod our heads, e.g. when we agree with someone, step from one foot to the other, when we want to speak, gesticulate with our hands to underline or clarify certain aspects. Furthermore, oral communication is embedded in actions - we talk while walking, shopping, on the way to the cinema, to school, etc. - and we talk when we are talking. All these movements (articulation movements, mimic movements, posture, gestures, running/cycling), their coordination, stability or frequency of occurrence in different situations are examined in the Motion Capture Lab.