MediaPipe Special Interest Group
The MediaPipe Special Interest Group is an ongoing exchange format for CRC 1252 members interested in video-based motion tracking, gesture analysis, sign language research, social gaze, and related methodological questions.
In phase III of CRC 1252, several projects in Area A focus on face-to-face communication and extend the notion of prosody beyond the oral-aural domain to include multimodal prosody. In this perspective, prominence can be signalled not only through speech, but also through hand, head, torso, eyebrow, and eye movements, as well as through social gaze and turn-taking behaviour.
The group focuses primarily on quantitative, computational, and computer-assisted workflows for multimodal research. This includes automatic and semi-automatic pipelines using tools such as MediaPipe, OpenPose, OpenFace, motion energy analysis, and related methods for extracting spatio-temporal indices of prominence from video and other multimodal data.
Annotation-based and more qualitative analyses are also relevant when they are supported by software tools or connected to computational workflows, for example for structured coding, comparison across methods, validation of automatically extracted features, or interpretation of visually prominent events. The emphasis of the group, however, is on reproducible, tool-supported, formal analysis pipelines.
CRC 1252 Research Context
The MediaPipe SIG is closely connected to CRC 1252's interest in how prominence is created, perceived, and negotiated in interaction. In the visual domain, this includes the relation between signal prominence and code prominence: some cues attract attention because of their physical salience, while others gain specific communicative force from their role in an interactional or linguistic system.
The group is also relevant for work on interactional prominence, that is, how speakers and signers become central in discourse and how control over interaction is established through combinations of speech, gesture, and social gaze. This makes the SIG relevant not only for technical tool discussions, but also for broader theoretical questions about embodied communication in CRC 1252, especially where automated or semi-automated methods can support analysis at different time scales and levels of granularity.
For a broader overview of CRC 1252 projects and members, see the SFB 1252 project overview.
Mailing List
There are currently 23 members subscribed to the mailing list.
New members can subscribe here:
Series Format
The group does not have a fixed recurring time slot. Additional meetings can be proposed between invited talks whenever there is interest.
Most Recent Invited Talk
Vadim Kimmelman (University of Bergen)
- Date: Wednesday, March 18, 2026
- Time: 10:00
- Topic: Experiences with pose-estimation workflows, with a particular focus on OpenPose
Kimmelman is a linguistics professor and principal investigator of an ERC project on universal properties of nonmanual gestures in sign languages.
Group History
Meetings so far:
- March 18, 2026, 10:00: Vadim Kimmelman (University of Bergen)
- November 19, 2025, 10:00: Anna Kuder et al.
- September 15, 2025, 14:00: Fabian Eckert, Aviad Albert et al.
- July 23, 2025, 11:30: Klym Myslyvyi, Janne Lorenzen et al.
- July 1, 2025, 11:00: Open discussion after PaPe and EnvisionBox