In the last few years, Neural Painting (NP) techniques became capable of producing extremely realistic artworks. This paper advances the state of the art in this emerging research domain by proposing the first approach for Interactive NP. Considering a setting where a user looks at a scene and tries to reproduce it on a painting, our objective is to develop a computational framework to assist the user's creativity by suggesting the next strokes to paint, that can be possibly used to complete the artwork. To accomplish such a task, we propose I-Paint, a novel method based on a conditional transformer Variational AutoEncoder (VAE) architecture with a two-stage decoder. To evaluate the proposed approach and stimulate research in this area, we also introduce two novel datasets. Our experiments show that our approach provides good stroke suggestions and compares favorably to the state of the art.

Interactive Neural Painting / Peruzzo, E. lia; Menapace, Willi; Goel, Vidit; Arrigoni, Federica; Tang, Hao; Xu, Xingqian; Chopikyan, Arman; Orlov, Nikita; Hu, Yuxiao; Shi, Humphrey; Sebe, Nicu; Ricci, Elisa. - In: COMPUTER VISION AND IMAGE UNDERSTANDING. - ISSN 1077-3142. - 235:(2023), pp. 10377801-10377810. [10.1016/j.cviu.2023.103778]

Interactive Neural Painting

Peruzzo, E. lia;Menapace, Willi;Arrigoni, Federica;Tang, Hao;Sebe, Nicu;Ricci, Elisa
2023-01-01

Abstract

In the last few years, Neural Painting (NP) techniques became capable of producing extremely realistic artworks. This paper advances the state of the art in this emerging research domain by proposing the first approach for Interactive NP. Considering a setting where a user looks at a scene and tries to reproduce it on a painting, our objective is to develop a computational framework to assist the user's creativity by suggesting the next strokes to paint, that can be possibly used to complete the artwork. To accomplish such a task, we propose I-Paint, a novel method based on a conditional transformer Variational AutoEncoder (VAE) architecture with a two-stage decoder. To evaluate the proposed approach and stimulate research in this area, we also introduce two novel datasets. Our experiments show that our approach provides good stroke suggestions and compares favorably to the state of the art.
2023
Peruzzo, E. lia; Menapace, Willi; Goel, Vidit; Arrigoni, Federica; Tang, Hao; Xu, Xingqian; Chopikyan, Arman; Orlov, Nikita; Hu, Yuxiao; Shi, Humphrey...espandi
Interactive Neural Painting / Peruzzo, E. lia; Menapace, Willi; Goel, Vidit; Arrigoni, Federica; Tang, Hao; Xu, Xingqian; Chopikyan, Arman; Orlov, Nikita; Hu, Yuxiao; Shi, Humphrey; Sebe, Nicu; Ricci, Elisa. - In: COMPUTER VISION AND IMAGE UNDERSTANDING. - ISSN 1077-3142. - 235:(2023), pp. 10377801-10377810. [10.1016/j.cviu.2023.103778]
File in questo prodotto:
File Dimensione Formato  
Elia-CVIU- July23-compressed.pdf

Solo gestori archivio

Descrizione: journal pre-proof
Tipologia: Post-print referato (Refereed author’s manuscript)
Licenza: Tutti i diritti riservati (All rights reserved)
Dimensione 597.95 kB
Formato Adobe PDF
597.95 kB Adobe PDF   Visualizza/Apri

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11572/388193
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 0
  • ???jsp.display-item.citation.isi??? 0
social impact