Investigating Communication and Social Behaviour Using Wearable Sensor Technology

The behaviour that we exhibit contributes to the message that is communicated to those that we are interacting with and can have an impact on how the message is conveyed and interpreted. Nonverbal behaviour is just as important to be aware of as well as what is being said, as the subtleties of behaviour can impact the outcome of interactions. Advancements in research technologies have allowed us the chance to investigate natural human behaviour is a variety of settings outside of the laboratory, however, some gaps in the understanding of behaviour exist. The aim of this thesis is to investigate communication and social interactions in a variety of settings, paying particular attention to the methods of data collection, specifically the use of wearable sensors, to investigate phenomenon from social psychology. The thesis aims to address three specific research questions; 1) if can we predict stress using a combination of nonverbal behavioural cues along with physiological measurements, 2) understand the factors affecting happiness and productivity in the workplace from features of communication taken from wearable sensors and 3) determine the stressors that can be characterised from communication patterns assessed through Call Detail Records and smartphone sensors. The studies presented here focus on the nonverbal aspects of communication that can be measured through wearable and sensing devices. In the three types of scenarios that are detailed in the different chapters, the interactions considered are face to face meetings in a one on one interaction, co-location within a defined space in an organisation and the communications of a widely dispersed community. The interactions are recorded by wearable devices such as the Affectiva Q sensor, the Sociometric Badge, and smartphones equipped with sensing capabilities in the form of the funf and P-OWL platforms for data recording, among other forms of data collection. Each of the studies included aspects of self reported assessments that were used as a ground truth measurement of affect: these were annotations of stress, self reports of fear of negative evaluation, self perception, positive and negative affect and stress, among others. The goal was to examine how to use digital traces of behavioural expressions to have a greater understanding of these interactions and how the way in which we interact with others has an impact on the individual. The work from this thesis adds to the existing literature on these various issues by addressing the research questions from a novel perspective. The studies found support for each of the research questions and by using a mixed methods approach and digital traces from wearable sensors gained insights into how communication impacted the individual, revealing the important aspects of communication and their effect on stress, productivity and well-being.