On the Use of Generative Adversarial Networks to Generate Face Images from Voice Neural Embeddings

Nowadays, the automatic detection of objects, either in images or in videos are topics of high interest in the development of technologies related to the deep learning approach, one of them is the field of forensic acoustics. Forensic acoustics is related to a security approach where the technology is used to try resolve policies’ issues. In this work, considering that Generative Adversarial Networks (GANs) have been used successfully in the reconstruction or generation images, the recognition of the face of people from its audio or video recordings has been studied using GANs. Due the novelty of this research, there are not many databases to evaluate the goodness of the models, so, an images-audio database has been created, containing both, images and audio of people who appear in videos of the YouTube platform. These images and corresponding audio embeddings have been used to train the proposed models based on GANs. The objective of this work is to generate the image of the face of a person considering only its audio voice signal as feature, that is, generate a face like the owner of the voice. The metric used to evaluate the efficiency of the proposed technique has been the “Peak Signal to Noise Ratio” metric (PSNR) which is able to determine if an image could be considered as a human face. Up to 28.39 dB of PSNR has been obtained when the images generated from its voice embeddings were evaluated, presenting up to 30% of relative improvement comparing to the same technique that use noise as feature.