Ask a Question
Ask Question Login
Corporate Training
  1. Community
  2. Artificial intelligence
  3. Question
Artificial intelligence

What's the reason for the mismatch between the definition of GAN loss function in two papers?

Asked by Alison Kelly May 18, 2022 643 views 1 answer
Share

About this question

I was trying to understand the loss function of GANs, but I found a little mismatch between different papers.

This is taken from the original GAN paper:

The adversarial modeling framework is most straightforward to apply when the models are both multilayer perceptrons. To learn the generator's distribution pg over data x, we define a prior on input noise variables pz(z), then represent a mapping to data space as G(z;θg), where G is a differentiable function represented by a multilayer perceptron with parameters θg. We also define a second multilayer perceptron D(x;θd) that outputs a single scalar. D(x) represents the probability that x came from the data rather than pg. We train D to maximize the probability of assigning the correct label to both training examples and samples from G. We simultaneously train G to minimize log(1−D(G(z))) :


In other words, D and G play the following two-player minimax game with value function V(G,D) :
minGmaxDV(D,G)=Ex∼pdata (x)[logD(x)]+Ez∼pz(z)[log(1−D(G(z)))]
Equation (1) in this version of the pix2pix paper

The objective of a conditional GAN can be expressed as

LcGAN(G,D)=Ex,y[logD(x,y)]+Ex,z[log(1−D(x,G(x,z))],

where G tries to minimize this objective against an adversarial D that tries to maximise it, i.e. G∗= arginmax LoGIN(G,D).


To test the importance of conditioning the discriminator, we also compare to an unconditional variant in which the discriminator does not observe x :

LGAN(G,D)=Ey[logD(y)]+Ex,z[log(1−D(G(x,z))].

Putting aside the fact that pix2pix is using conditional GAN, which introduces a second term y, the 2 formulas are quite resemble, except that in the pix2pix paper, they try to get minimax of LcGAN(G,D), which is defined to be Ex,y[...]+Ex,z[...], whereas in the original paper, they define minmax(G,D)=E[...]+E[...].


I am not coming from a good math background, so I am quite confused. I'm not sure where the mistake is, but assuming that E is an expectation (correct me if I'm wrong), the version in pix2pix makes more sense to me, although I think it's quite less likely that Goodfellow could make this mistake in his amazing paper. Maybe there's no mistake at all and it's me who does not understand them correctly.

Your answer

1 Answer

More Artificial intelligence discussions

Learn & Explore

Free tutorials and interview questions from industry experts — learn the skill, then get ready to prove it.

Latest Artificial Intelligence Blogs

Guides, tips and career advice on Artificial Intelligence from JanBask experts.