A Computational Model of Social Signalling

 

By Alex Pentland

(Professor, MIT Media Lab, USA)

Review by:

Adrian ION
Vienna University of Technology

 

Motivated by the desire to increase the quality of collaboration and cooperation between humans, Professor Alex Pentland and his Human Dynamics research group at MIT have looked into the problem of understanding and predicting social aspects that seem to play an important role in human interaction, i.e., social context. First steps toward identifying social context in human communication have been taken by the development of three socially aware platforms that objectively measure several aspects of social context.

 

One of these platforms uses non-linguistic social signals and has been found to be particularly powerful for analysing and predicting human behaviour, sometimes exceeding even expert human capabilities. Social signals are non-linguistic signals measured by analysing the person's tone of voice, facial movement, or gesture. To quantify these social signals, texture-like measures have been developed for four types of social signalling: activity level, engagement, stress, and mirroring. Activity level is how much you participate in the conversation and is measured by the percentage of speaking time during a conversation. Engagement is how involved a person is in the current interaction, i.e., whether he or she is driving the conversation, setting the tone, etc. It is measured by looking at the influence speakers have on each other, i.e., when two people are acting, their individual turn-taking dynamics influence each other. Stress is the variation in prosodic emphasis and is measured by looking at the standard deviation of the formant frequency and spectral entropy (base frequency and frequency spread of one's voice during the conversation). Mirroring occurs when one participant subconsciously copies another participant's prosody and gesture. It is considered a signal of empathy and is measured by looking at the frequency of short interjections (“Uh-huh”) and back-and-forth exchanges typically consisting of single words (“OK?”, “OK!”).

 

The presented social signalling measurements have been incorporated into three socially aware communication systems (a badge, a PDA, and a mobile phone), and a set of experiments has been done. The experiments consisted in predicting the outcome in the following situations: who would exchange business cards at a meeting; which couples would exchange phone numbers at a bar; who would come out ahead in a negotiation; who was a connector within a work group; and a range of subjective judgements, including whether or not a person felt a negotiation was honest and fair or a conversation was interesting. After excluding cases where not enough data was available to make a decision, an average accuracy of almost 90 percent was achieved. If the system was asked to give an answer independent of the amount of available data, an accuracy of around 80 percent was observed.

More on this work can be found at:

http://web.media.mit.edu/~sandy/

Sandy Pentland is shown at left interacting with Ron Caneel and Nathan Eagle, all wearing badge systems containing a microphone, accelerometer (motion sensor), and IR tag, for analyzing social interactions.