Potential inconsistencies in calculation of gradient penalty between code and ArXiv paper about improved_wgan_training HOT 4 CLOSED

________________________________ From: Mark Wronkiewicz <[email protected]> Sent: Wednesday, July 26, 2017 2:41:54 PM To: igul222/improved_wgan_training Cc: Subscribed Subject: [igul222/improved_wgan_training] Potential inconsistencies in calculation of gradient penalty between code and ArXiv paper (#39) I could be wrong, but it seems like the calculation for the gradient penalty is not the same across different code examples in this repo. In the paper, I believe the calculation is shown in line 6 in Algorithm 1 (page 4 in ArXiv paper) -- that line suggests the second of the 2 options is correct. However, most code examples seem to use the first option below. Option 1 In gan_mnist.py (Line 143-144), gan_64x64.py (495-496), gan_language.py (104-105), gan_cifar.py (130-131), and gan_cifar_resnet.py (260-261: differences = fake_data - real_data interpolates = real_data + (alpha*differences) # After rearranging, equivalent to: # real_data + alpha*fake_data - alpha*real_data Option 2 In gan_toy.py (Line 77) and ArXiv paper (Algorithm 1, line 6 on page 4): interpolates = alpha*real_data + ((1-alpha)*fake_data) # After rearranging, equivalent to: # fake_data + alpha*real_data - alpha*fake_data real_data and fake_data seem to be transposed between the two options. Am I missing something? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub<#39>, or mute the thread<https://github.com/notifications/unsubscribe-auth/ABBP7tjqcU-vf14r3LHhYVgVHR2KohHyks5sR7KigaJpZM4Okhsz>.

________________________________ From: Mark Wronkiewicz <[email protected]> Sent: Thursday, July 27, 2017 11:01:44 AM To: igul222/improved_wgan_training Cc: Ishaan Gulrajani; Comment Subject: Re: [igul222/improved_wgan_training] Potential inconsistencies in calculation of gradient penalty between code and ArXiv paper (#39) Had to think about it a while, but I think I get it :) -- can you spot check this geometric interpretation? Basically, the differences vector goes from the real_data to the fake_data. Because you're adding on a random proportion of this vector to the real_data, you're essentially getting some random point on the line between the real_data and fake_data vectors. And because it's random between 0 and 1, it doesn't matter if you instead start from fake_data and traverse towards real_data -- you'll still end up at some random point on the line between the two. — You are receiving this because you commented. Reply to this email directly, view it on GitHub<#39 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/ABBP7n8HWC0umKHyOV0jT1-dREIJdTbFks5sSNCIgaJpZM4Okhsz>.