Guiding the matrix: conditional GANs and meta-learning for image synthesis

Abstract

Conditioning Generative Adversarial Networks (GANs), in order to guide them, is a fundamental step to better understand the behaviour of GANs and to greatly improve their results and usability. More specifically, this means feeding additional information into the network in order to control the generated samples. The problems addressed in this thesis will cover a broad range of topics like shoe colorization, few shot colorization, multi-domain image-to-image translation, meta-learning, attention transfer and GANs compression. In particular, will be described: firstly, a system able to automatically colorize shoe models starting from a blank or partially colored 3d model; secondly, a new architecture called MetalGAN that combines meta-learning and conditional GANs for few-shot image colorization; thirdly, an extension of MetalGAN that tackles the problem of multi-domain image-to-image translation and outperforms existing methods; finally, a system where attention maps can be used as knowledge to be transferred in a teacher-student paradigm for image-to-image translation improving the student performance. Each of them deals with conditioning an adversarial network in different ways. For each system both qualitative and quantitative experimental results will be presented.