bug in calc_gradient_penalty?

[lezhang-thu]

wgan-gp/gan_language.py

Line 159 in ae47a18

gradient_penalty = ((gradients.norm(2, dim=1) - 1) ** 2).mean() * LAMBDA

Right before this line, it should be added "gradients = gradients.view(BATCH_SIZE, -1)"?

[t-ae]

It's OK in gan_mnist.py because real_data and fake_data has shape [BATCH_SZIE, OUTPUT_DIM], and Discriminator reshapes input at first.

wgan-gp/gan_mnist.py

Line 100 in ae47a18

input = input.view(-1, 1, 28, 28)

But, in gan_cifar10.py, Discriminator requires input tensor to have rank 4. It means gradients also has rank 4.

wgan-gp/gan_cifar10.py

Lines 89 to 93 in ae47a18

	def forward(self, input):
	output = self.main(input)
	output = output.view(-1, 4*4*4*DIM)
	output = self.linear(output)
	return output

So computation of gradient_penalty is not correct.

In original(?) implementation Discriminator requires rank 2 tensor.
https://github.com/igul222/improved_wgan_training/blob/master/gan_cifar.py#L71

In gan_language.py, original Discriminator takes rank 3 tensor. But norm is computed along two axes.
https://github.com/igul222/improved_wgan_training/blob/master/gan_language.py#L107