We introduce a novel approach for sequence decoding, Discriminative Adversarial Search (DAS), which has the desirable properties of alleviating the effects of exposure bias without requiring external metrics. Inspired by Generative Adversarial Networks (GANs), wherein a discriminator is used to improve the generator, our method differs from GANs in that the generator parameters are not updated at training time and the discriminator is only used to drive sequence generation at inference time. We investigate the effectiveness of the proposed approach on the task of Abstractive Summarization: the results obtained show that DAS improves over the stateof- the-art methods, with further gains obtained via discriminator retraining. Moreover, we show how DAS can be effective for cross-domain adaptation. Finally, all results reported are obtained without additional rule-based filtering strategies, commonly used by the best performing systems available: this indicates that DAS can effectively be deployed without relying on post-hoc modifications of the generated outputs.

Discriminative adversarial search for abstractive summarization / Scialom, Thomas; Dray, Paul-Alexis; Lamprier, Sylvai; Piwowarski, Benjamin; Staiano, Jacopo. - 116:(2020), pp. 8513-8522. (Intervento presentato al convegno ICML 2020 tenutosi a Virtual, Online nel 2020).

Discriminative adversarial search for abstractive summarization

Staiano, Jacopo
Ultimo
2020-01-01

Abstract

We introduce a novel approach for sequence decoding, Discriminative Adversarial Search (DAS), which has the desirable properties of alleviating the effects of exposure bias without requiring external metrics. Inspired by Generative Adversarial Networks (GANs), wherein a discriminator is used to improve the generator, our method differs from GANs in that the generator parameters are not updated at training time and the discriminator is only used to drive sequence generation at inference time. We investigate the effectiveness of the proposed approach on the task of Abstractive Summarization: the results obtained show that DAS improves over the stateof- the-art methods, with further gains obtained via discriminator retraining. Moreover, we show how DAS can be effective for cross-domain adaptation. Finally, all results reported are obtained without additional rule-based filtering strategies, commonly used by the best performing systems available: this indicates that DAS can effectively be deployed without relying on post-hoc modifications of the generated outputs.
2020
37th International Conference on Machine Learning
Stroudsburg, PA, USA
International Machine Learning Society (IMLS); Curran Associates
9781713821120
Scialom, Thomas; Dray, Paul-Alexis; Lamprier, Sylvai; Piwowarski, Benjamin; Staiano, Jacopo
Discriminative adversarial search for abstractive summarization / Scialom, Thomas; Dray, Paul-Alexis; Lamprier, Sylvai; Piwowarski, Benjamin; Staiano, Jacopo. - 116:(2020), pp. 8513-8522. (Intervento presentato al convegno ICML 2020 tenutosi a Virtual, Online nel 2020).
File in questo prodotto:
File Dimensione Formato  
3524938.3525731.pdf

Solo gestori archivio

Tipologia: Versione editoriale (Publisher’s layout)
Licenza: Tutti i diritti riservati (All rights reserved)
Dimensione 2.2 MB
Formato Adobe PDF
2.2 MB Adobe PDF   Visualizza/Apri

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11572/363002
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 8
  • ???jsp.display-item.citation.isi??? ND
  • OpenAlex ND
social impact