Mastering PyTorch by Ashish Ranjan Jha

Author:Ashish Ranjan Jha [Jha, Ashish Ranjan]
Language: eng
Format: epub
Tags: Machine learning; computer vision; hugging face; fastai; nlp python; natural language processing; CNNs
Publisher: Packt Publishing
Published: 2023-06-19T00:00:00+00:00

Defining the generator and discriminator networks

As mentioned earlier, GANs are composed of two components – the generator and the discriminator. Both of these are essentially neural networks. Generators and discriminators with different neural architectures produce different types of GANs. For example, DCGANs purely have CNNs as the generator and discriminator. You can find a list of different types of GANs along with their PyTorch implementations at [9.1] .

For any GAN that is used to generate some kind of real data, the generator usually takes random noise as input and produces an output with the same dimensions as the real data. We call this generated output fake data. The discriminator, on the other hand, works as a binary classifier. It takes in the generated fake data and the real data (one at a time) as input and predicts whether the input data is real or fake. Figure 9 .1 shows a diagram of the overall GAN model schematic:

Figure 9 .1 – A GAN schematic The discriminator network is optimized like any binary classifier, that is, using the binary cross-entropy function. Therefore, the discriminator model's motivation is to correctly classify real images as real and fake images as fake. The generator network has quite the opposite motivation. The generator loss is mathematically expressed as -log(D(G(x))), where x is random noise inputted into the generator model, G; G(x) is the generated fake image by the generator model; and D(G(x)) is the output probability of the discriminator model, D, that is, the probability of the image being real.

Therefore, the generator loss is minimized when the discriminator thinks that the generated fake image is real. Essentially, the generator is trying to fool the discriminator in this joint optimization problem.

In execution, these two loss functions are backpropagated alternatively. That is, at every iteration of training, first, the discriminator is frozen, and the parameters of the generator networks are optimized by backpropagating the gradients from the generator loss.

Then, the tuned generator is frozen while the discriminator is optimized by backpropagating the gradients from the discriminator loss. This is what we call joint optimization. It has also been referred to as being equivalent to a two-player Minimax game in the original GAN paper [9.2] .

Download

Copyright Disclaimer:
This site does not store any files on its server. We only index and link to content provided by other sites. Please contact the content providers to delete copyright contents if any and email us, we'll remove relevant links or contents immediately.