Mood inference with mobile sensing data has been studied in ubicomp literature over the last decade. This inference enables context-aware and personalized user experiences in general mobile apps and valuable feedback and interventions in mobile health apps. However, even though model generalization issues have been highlighted in many studies, the focus has always been on improving the accuracies of models using different sensing modalities and machine learning techniques, with datasets collected in homogeneous populations. In contrast, less attention has been given to studying the performance of mood inference models to assess whether models generalize to new countries. In this study, we collected a mobile sensing dataset with 329K self-reports from 678 participants in eight countries (China, Denmark, India, Italy, Mexico, Mongolia, Paraguay, UK) to assess the effect of geographical diversity on mood inference models. We define and evaluate country-specific (trained and tested within a country), continent-specific (trained and tested within a continent), country-agnostic (tested on a country not seen on training data), and multi-country (trained and tested with multiple countries) approaches trained on sensor data for two mood inference tasks with population-level (non-personalized) and hybrid (partially personalized) models. We show that partially personalized country-specific models perform the best yielding area under the receiver operating characteristic curve (AUROC) scores of the range 0.78-0.98 for two-class (negative vs. positive valence) and 0.76-0.94 for three-class (negative vs. neutral vs. positive valence) inference. Further, with the country-agnostic approach, we show that models do not perform well compared to country-specific settings, even when models are partially personalized. We also show that continent-specific models outperform multi-country models in the case of Europe. Overall, we uncover generalization issues of mood inference models to new countries and how the geographical similarity of countries might impact mood inference.

Generalization and Personalization of Mobile Sensing-Based Mood Inference Models: An Analysis of College Students in Eight Countries / Meegahapola, L; Droz, W; Kun, P; de Gotzen, A; Nutakki, C; Diwakar, S; Correa, Sr; Song, Dl; Xu, H; Bidoglia, M; Gaskell, G; Chagnaa, A; Ganbold, A; Zundui, T; Caprini, C; Miorandi, D; Hume, A; Zarza, Jl; Cernuzzi, L; Bison, I; Britez, Mr; Busso, M; Chenu-Abente, R; Gunel, C; Giunchiglia, F; Schelenz, L; Gatica-Perez, D. - In: PROCEEDINGS OF THE ACM ON INTERACTIVE, MOBILE, WEARABLE AND UBIQUITOUS TECHNOLOGIES. - ISSN 2474-9567. - 6:4(2022), pp. 1-32. [10.1145/3569483]

Generalization and Personalization of Mobile Sensing-Based Mood Inference Models: An Analysis of College Students in Eight Countries

Caprini, C;Miorandi, D;Hume, A;Cernuzzi, L;Bison, I;Busso, M;Chenu-Abente, R;Gunel, C;Giunchiglia, F;
2022-01-01

Abstract

Mood inference with mobile sensing data has been studied in ubicomp literature over the last decade. This inference enables context-aware and personalized user experiences in general mobile apps and valuable feedback and interventions in mobile health apps. However, even though model generalization issues have been highlighted in many studies, the focus has always been on improving the accuracies of models using different sensing modalities and machine learning techniques, with datasets collected in homogeneous populations. In contrast, less attention has been given to studying the performance of mood inference models to assess whether models generalize to new countries. In this study, we collected a mobile sensing dataset with 329K self-reports from 678 participants in eight countries (China, Denmark, India, Italy, Mexico, Mongolia, Paraguay, UK) to assess the effect of geographical diversity on mood inference models. We define and evaluate country-specific (trained and tested within a country), continent-specific (trained and tested within a continent), country-agnostic (tested on a country not seen on training data), and multi-country (trained and tested with multiple countries) approaches trained on sensor data for two mood inference tasks with population-level (non-personalized) and hybrid (partially personalized) models. We show that partially personalized country-specific models perform the best yielding area under the receiver operating characteristic curve (AUROC) scores of the range 0.78-0.98 for two-class (negative vs. positive valence) and 0.76-0.94 for three-class (negative vs. neutral vs. positive valence) inference. Further, with the country-agnostic approach, we show that models do not perform well compared to country-specific settings, even when models are partially personalized. We also show that continent-specific models outperform multi-country models in the case of Europe. Overall, we uncover generalization issues of mood inference models to new countries and how the geographical similarity of countries might impact mood inference.
2022
4
Meegahapola, L; Droz, W; Kun, P; de Gotzen, A; Nutakki, C; Diwakar, S; Correa, Sr; Song, Dl; Xu, H; Bidoglia, M; Gaskell, G; Chagnaa, A; Ganbold, A; Zundui, T; Caprini, C; Miorandi, D; Hume, A; Zarza, Jl; Cernuzzi, L; Bison, I; Britez, Mr; Busso, M; Chenu-Abente, R; Gunel, C; Giunchiglia, F; Schelenz, L; Gatica-Perez, D
Generalization and Personalization of Mobile Sensing-Based Mood Inference Models: An Analysis of College Students in Eight Countries / Meegahapola, L; Droz, W; Kun, P; de Gotzen, A; Nutakki, C; Diwakar, S; Correa, Sr; Song, Dl; Xu, H; Bidoglia, M; Gaskell, G; Chagnaa, A; Ganbold, A; Zundui, T; Caprini, C; Miorandi, D; Hume, A; Zarza, Jl; Cernuzzi, L; Bison, I; Britez, Mr; Busso, M; Chenu-Abente, R; Gunel, C; Giunchiglia, F; Schelenz, L; Gatica-Perez, D. - In: PROCEEDINGS OF THE ACM ON INTERACTIVE, MOBILE, WEARABLE AND UBIQUITOUS TECHNOLOGIES. - ISSN 2474-9567. - 6:4(2022), pp. 1-32. [10.1145/3569483]
File in questo prodotto:
File Dimensione Formato  
2022 Generalization and Personalization.pdf

Solo gestori archivio

Tipologia: Versione editoriale (Publisher’s layout)
Licenza: Tutti i diritti riservati (All rights reserved)
Dimensione 2.39 MB
Formato Adobe PDF
2.39 MB Adobe PDF   Visualizza/Apri

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11572/391369
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 11
  • ???jsp.display-item.citation.isi??? 7
social impact