We describe a system that can associate images with English proverbs. We start from a corpus of proverbs, harvest related images from the web and use this data to train two variants of a convolutional neural network. We then collect a small set of annotations, and use these to combine the outputs of the two networks into a single prediction for each input image. We carry out feature selection experiments on a set of features derived from the images and from the predicted proverbs, and demonstrate that the metaphoricity of the proverbs plays a significant role in classification accuracy. An empirical evaluation with human raters confirms the system's ability to abstract from the raw bits in the images and to learn meaningful, non-trivial associations.

A proverb is worth a thousand words: learning to associate images with proverbs / Gözde, Özbal; Pighin, Daniele; Strapparava, Carlo. - (2019), pp. 2515-2521. ((Intervento presentato al convegno 41st Annual Conference of the Cognitive Science Society (CogSci’19) tenutosi a Montreal, Quebec, Canada nel July 2019.

A proverb is worth a thousand words: learning to associate images with proverbs

Gözde Özbal;Daniele Pighin;Carlo Strapparava
2019

Abstract

We describe a system that can associate images with English proverbs. We start from a corpus of proverbs, harvest related images from the web and use this data to train two variants of a convolutional neural network. We then collect a small set of annotations, and use these to combine the outputs of the two networks into a single prediction for each input image. We carry out feature selection experiments on a set of features derived from the images and from the predicted proverbs, and demonstrate that the metaphoricity of the proverbs plays a significant role in classification accuracy. An empirical evaluation with human raters confirms the system's ability to abstract from the raw bits in the images and to learn meaningful, non-trivial associations.
Proceedings of the 41st Annual Conference of the Cognitive Science Society (CogSci’19)
USA
Cognitive Science Society
0-9911967-7-5
Gözde, Özbal; Pighin, Daniele; Strapparava, Carlo
A proverb is worth a thousand words: learning to associate images with proverbs / Gözde, Özbal; Pighin, Daniele; Strapparava, Carlo. - (2019), pp. 2515-2521. ((Intervento presentato al convegno 41st Annual Conference of the Cognitive Science Society (CogSci’19) tenutosi a Montreal, Quebec, Canada nel July 2019.
File in questo prodotto:
Non ci sono file associati a questo prodotto.

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11572/343996
 Attenzione

Attenzione! I dati visualizzati non sono stati sottoposti a validazione da parte dell'ateneo

Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus ND
  • ???jsp.display-item.citation.isi??? ND
social impact