Test set used as validation to choose best model about atlasnet HOT 9 CLOSED

Hi @lynetcha ,

Regarding the results that you get with run_AE_AtlasNet.py, I think that the numbers given in the README are taken from the paper. In the latter, a sampling of 2500 points is used to compute the Chamfer distance, whereas the default number of samples in run_AE_AtlasNet.py is 30000, probably in order to get a more accurate result.

Since the Chamfer distance is computed thanks to a pairwise assignation, the more points you sample, the closer the closest point will be for every point. As a consequence, the Chamfer distance can only improve. In order to produce results that are close to the ones you get in the README, I advise you to set the parameter --gen_points to 2500.

Best regards

from atlasnet.

yes indeed. this code is a cleaned version of the more heavy one i developped during the project. in my version, i save a bunch of relevant networks for testing purposes namely last epoch, best train and best test. to reproduce the paper, the network from the last epoch has to be used indeed. i'll clean that tomorrow. thanks for the feedback!

from atlasnet.

Thank you!

from atlasnet.

One more thing, I assume the trained models currently linked are not the ones used in the paper since I ran those models (using inference/run_AE_AtlasNet.py) and the val_loss is different than what is reported in the paper and the Metro loss is 0. Can you please link the models used in the paper? Thanks.

from atlasnet.

Hi, yes I was rushed for the deadline so I retrained the networks with a few added tricks after the deadline (I think learning rate decay + longer training). So the network available are the same as the one in the paper but they should just work slightly better (results are in the README).
The metro distance code is not included in the repo. it outputs 0 because it is not evaluated. you can find it here : https://github.com/RobotLocomotion/meshConverters/tree/master/vcglib/apps/metro

from atlasnet.

Can you double-check that this is the correct network? I ran run_AE_AtlasNet.py using the linked trained network and the results I am getting are significantly better than the ones in the README (1.07e-3 instead of 1.47e-3).
Below are the per-class results I have
cellphone 0.000825
watercraft 0.000955
monitor 0.001159
car 0.001052
couch 0.001085
cabinet 0.001094
lamp 0.001917
plane 0.000610
speaker 0.001822
table 0.001047
chair 0.001106
bench 0.000811
firearm 0.000435
Mean 0.001071

from atlasnet.

I can't do it in the short term, but i'll do it eventually. i you need a quick answer, the best way would be to retrain a network, otherwise i'll get back to you in ~1-2 weeks.

from atlasnet.

I have been having trouble retraining the network due the batchnorm instabilities. I will see if I can fix that issue. Thanks!

from atlasnet.

Ok, I will try that. Thanks @palanglois!

from atlasnet.