Comments (15)
thank @jorge-pessoa for the great code, and thank @PK15946 @serkansulun for the different solution. I would like to discuss the underlying reason of the NAN problem.
when reproduce the problem, I notice the nan first appear in backward in very early iteration. There is something wrong with the autograd of ms-ssim. So I check the formula.
And there is. cs**a
and ssim**a
, where a is a given parameter, have the 1/cs**(a-1)
format in backward. the value of cs or ssim may be very close to zero, and the backward value is +inf.
To avoid nagetive zero, or very small positive value, using normalize is ok, and switch to msssim after enough iteration is also work. For me, I use cs.clamp(min=0.00001)
from pytorch-msssim.
Hi @jorge-pessoa , thanks a lot for your code! So we have to train with other loss first then switch to ms-ssim to avoid the bug yet?
from pytorch-msssim.
SSIM value can be smaller than zero, as discussed here.
Taking its power with a fractional number yields NaNs.
I avoid it by normalizing the values between 0 and 1:
ret = (ret + 1) / 2
cs = (cs + 1) /2
Let me know if it's a bad idea.
from pytorch-msssim.
Hello@jorge-pessoa, thank for your excellent code! I am interested in the file of max_ssim.py. However, I encountered a question that when I replace the 'pytorch_msssim.SSIM()' to 'pytorch_msssim.MSSSIM()' , it is hard to converge and cannot get the max quality score. I also try some other optimizers and learning rates but they are all failed. Do you have any idea about it? I am looking forward getting your reply.
from pytorch-msssim.
@KimiHanWei For dev branch, when I run with SSIM, it takes 78 iterations and with MSSSIM it takes 135 to reach a value of 0.95. Also, in both trials, the values are increasing nicely.
from pytorch-msssim.
@ssulun16 thanks a lot, when I run with SSIM, it takes 80 iterations to reach 0.95 but with MSSSIM it traps in a value of 0.25, I just hide the codes from 120 to 121 in 'init.py', since it reports an error 'zero-dimensional tensor (at position 0) cannot be concatenated'. Have you made any change to the source code? Hopefully get your answer.
from pytorch-msssim.
from pytorch-msssim.
from pytorch-msssim.
@PK15946 If you are using PyTorch 0.4 you can also skip any batch that yields a NAN result by checking if the results from the msssim function is nan using torch.isnan() method.
@ssulun16 Thank you for your observation, you are indeed correct; I definitely missed that point. I will investigate if the normalization you proposed doesn't change the results in a meaningful way. In this case I will integrate your suggestion in the repository (you are welcome to submit a pull request if you would like to contribute!).
I also noted a few cases where the current code in the dev branch is calculating a slightly lower MS-SSIM than expected, which I will take a look at in the following days.
from pytorch-msssim.
@jorge-pessoa I'm using this in my research so I have made plenty of changes but I made a pull request (I hope, I'm a newbie in Github).
Also unrelated but, I couldn't find your email so I wanted to try my chances here. I'm interested in pursuing a PhD in Lisbon so if you send me an email (address written in my profile), I would be very happy to ask you a few questions.
from pytorch-msssim.
Hi, @jorge-pessoa. Thanks for this code, very useful. Just some questions there :
- was the bug in this issue corrected ?
- can we use the SSIM programmed in the dev branch with torch 0.4 or only MSSSIM ?
Thanks a lot ! :)
from pytorch-msssim.
Hello,
@SalzingerJules you can check the Pull Request submited by @ssulun at #3 which solves this specific problem (note that the normalization is commented). However there are a few things I would like to change on the pull request before integrating it into the dev branch and I didn't have time yet to do so. Hopefully in the next few days I can merge everything and switch the main branch to the 0.4 version.
Both the SSIM and MS-SSIM should work properly in pytorch 0.4 @ dev branch.
Cheers
from pytorch-msssim.
The current version on the dev branch can now receive the parameter normalize=<True|False>. This can be set to true for the first few iterations of training or during all the training, avoiding NaN results. It should be useful to train unstable models using the MS-SSIM metric. The changes are still being tested however they should fix the problem reported on this issue.
from pytorch-msssim.
This issue is now resolved on the master branch with the commit 6378855 , by introducing the normalization flag.
from pytorch-msssim.
@jorge-pessoa ,Setting the parameter normalize really works,Thanks
from pytorch-msssim.
Related Issues (20)
- nan problem HOT 2
- Output is out of range HOT 2
- Bug in your MSSSIM implementation HOT 3
- Is the threshold necessary? HOT 1
- Inquiry about standard deviations HOT 1
- ssim on mnist HOT 1
- Index map HOT 1
- Contrast Sensitivity (CS) for batch size > 1 HOT 1
- Typo in calculation? HOT 2
- Big in SSIM implementation, don't use this code for perceptual quality estimation HOT 3
- SSIM result is different from skimage.measure.compare_ssim HOT 1
- parameter misspelled in readme HOT 1
- Usage of this loss in the binary segmentation
- SSIM loss output range
- Artifacts on color images HOT 5
- MS-SSIM Usage HOT 1
- Expected object of scalar type Double but got scalar type Float for argument #2 'weight' HOT 2
- Possible mistake in your implementation? HOT 3
- functools.lru_cache can be used for saving the window HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pytorch-msssim.