About this question
I was trying to understand the loss function of GANs, but I found a little mismatch between different papers.
This is taken from the original GAN paper:
The adversarial modeling framework is most straightforward to apply when the models are both multilayer perceptrons. To learn the generator's distribution pg over data x, we define a prior on input noise variables pz(z), then represent a mapping to data space as G(z;θg), where G is a differentiable function represented by a multilayer perceptron with parameters θg. We also define a second multilayer perceptron D(x;θd) that outputs a single scalar. D(x) represents the probability that x came from the data rather than pg. We train D to maximize the probability of assigning the correct label to both training examples and samples from G. We simultaneously train G to minimize log(1−D(G(z))) :
In other words, D and G play the following two-player minimax game with value function V(G,D) :
minGmaxDV(D,G)=Ex∼pdata (x)[logD(x)]+Ez∼pz(z)[log(1−D(G(z)))]
Equation (1) in this version of the pix2pix paper
The objective of a conditional GAN can be expressed as
LcGAN(G,D)=Ex,y[logD(x,y)]+Ex,z[log(1−D(x,G(x,z))],
where G tries to minimize this objective against an adversarial D that tries to maximise it, i.e. G∗= arginmax LoGIN(G,D).
To test the importance of conditioning the discriminator, we also compare to an unconditional variant in which the discriminator does not observe x :
LGAN(G,D)=Ey[logD(y)]+Ex,z[log(1−D(G(x,z))].Putting aside the fact that pix2pix is using conditional GAN, which introduces a second term y, the 2 formulas are quite resemble, except that in the pix2pix paper, they try to get minimax of LcGAN(G,D), which is defined to be Ex,y[...]+Ex,z[...], whereas in the original paper, they define minmax(G,D)=E[...]+E[...].
I am not coming from a good math background, so I am quite confused. I'm not sure where the mistake is, but assuming that E is an expectation (correct me if I'm wrong), the version in pix2pix makes more sense to me, although I think it's quite less likely that Goodfellow could make this mistake in his amazing paper. Maybe there's no mistake at all and it's me who does not understand them correctly.