HUMSS - Evie Malaia

Quantifying sign language signalling in the communication chain using explainable AI

Sign languages are communication systems that are equivalent to spoken languages in capacity for information transfer, but use dynamic visual signal. Comparison of physical and communicative (informative) signal across modalities is complicated by higher (spatial + temporal) dimensionality of the sign language signal as compared to speech. The talk will discuss a variety of approaches used to characterize sign languages based on linguistic and physical data (motion capture, video, and microdoppler radar signal), and present new findings on sign language processing and production obtained using explainable AI methods.