In this paper, we present a novel framework that can produce a visual description of a tourist attraction by choosing the most diverse pictures from community-contributed datasets, that describe different details of the queried location. The main strength of the proposed approach is its flexibility that permits to filter out non-relevant images, and to obtain a reliable set of diverse and relevant images by first clustering similar images according to their textual descriptions and their visual content, and then extracting images from different clusters according to a measure of user's credibility. Clustering is based on a two-step process where textual descriptions are used first, and the clusters are then refined according to the visual features. The degree of diversification can be further increased by exploiting users' judgments on the results produced by the proposed algorithm through a novel approach, where users not only provide a Relevance Feedback, but also a Diversity Feedback. Experimental results performed on the MediaEval 2015 "Retrieving Diverse Social Images" dataset show that the proposed framework can achieve very good performance both in the case of automatic retrieval of diverse images, and in the case of the exploitation of the users' feedback. The effectiveness of the proposed approach has been also confirmed by a small case study involving a number of real users.

Multimodal retrieval with diversification and relevance feedback for tourist attraction images / Dang-Nguyen, Duc-Tien; Piras, Luca; Giacinto, Giorgio; Boato, Giulia; De Natale, Francesco G. B.. - In: ACM TRANSACTIONS ON MULTIMEDIA COMPUTING, COMMUNICATIONS AND APPLICATIONS. - ISSN 1551-6857. - 2017, 13:4(2017), pp. 49.1-49.24. [10.1145/3103613]

Multimodal retrieval with diversification and relevance feedback for tourist attraction images

Dang-Nguyen, Duc-Tien;Piras, Luca;Giacinto, Giorgio;Boato, Giulia;De Natale, Francesco G. B.
2017-01-01

Abstract

In this paper, we present a novel framework that can produce a visual description of a tourist attraction by choosing the most diverse pictures from community-contributed datasets, that describe different details of the queried location. The main strength of the proposed approach is its flexibility that permits to filter out non-relevant images, and to obtain a reliable set of diverse and relevant images by first clustering similar images according to their textual descriptions and their visual content, and then extracting images from different clusters according to a measure of user's credibility. Clustering is based on a two-step process where textual descriptions are used first, and the clusters are then refined according to the visual features. The degree of diversification can be further increased by exploiting users' judgments on the results produced by the proposed algorithm through a novel approach, where users not only provide a Relevance Feedback, but also a Diversity Feedback. Experimental results performed on the MediaEval 2015 "Retrieving Diverse Social Images" dataset show that the proposed framework can achieve very good performance both in the case of automatic retrieval of diverse images, and in the case of the exploitation of the users' feedback. The effectiveness of the proposed approach has been also confirmed by a small case study involving a number of real users.
2017
4
Dang-Nguyen, Duc-Tien; Piras, Luca; Giacinto, Giorgio; Boato, Giulia; De Natale, Francesco G. B.
Multimodal retrieval with diversification and relevance feedback for tourist attraction images / Dang-Nguyen, Duc-Tien; Piras, Luca; Giacinto, Giorgio; Boato, Giulia; De Natale, Francesco G. B.. - In: ACM TRANSACTIONS ON MULTIMEDIA COMPUTING, COMMUNICATIONS AND APPLICATIONS. - ISSN 1551-6857. - 2017, 13:4(2017), pp. 49.1-49.24. [10.1145/3103613]
File in questo prodotto:
File Dimensione Formato  
a49-dang-nguyen.pdf

Solo gestori archivio

Tipologia: Post-print referato (Refereed author’s manuscript)
Licenza: Tutti i diritti riservati (All rights reserved)
Dimensione 1.43 MB
Formato Adobe PDF
1.43 MB Adobe PDF   Visualizza/Apri

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11572/195644
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 20
  • ???jsp.display-item.citation.isi??? 14
  • OpenAlex ND
social impact