Toward Open-World Multimedia Forensics Through Media Signature Encoding

Countering image and video manipulations is getting more and more relevant in several fields such as investigation, intelligence and forensics. Multimedia forensics researchers keep developing new tools and updating available detectors to discriminate the processing the media has been subjected to. While these tools can be utilized efficiently in controlled settings, they are generally unreliable in open-world scenarios where the investigated material may have been subjected to several unknown manipulations. In this paper, we present a novel framework to discriminate different toolchains of media manipulation and processing. We introduce the concept of media signature encoding to map image and video contents to latent spaces where media produced by similar processing toolchains cluster together. We demonstrate that this property still holds for toolchain that are not known when building the encoder, expanding the range of applications for our framework to open-world contexts where fore...