validation loss explodes about atlasnet HOT 8 CLOSED

I'll get back to you tomorrow, I need to run some tests. Thanks for pointing this out

from atlasnet.

It works fine for me. I need to test more things, it might be an issue with the data online. I'll get back to you on monday.

from atlasnet.

Ok, looking forward to your reply.

from atlasnet.

hi! i've started running the code from scratch. it's been going well for the first epochs. My guess is that your chamfer distance has issues. Did you compile the chamfer distance as indicated in the Readme? what version of pytorch are you using ? can you check the output of the chamfer distance on a toy example (cd nndistance; python test.py)?

from atlasnet.

That's odd, I did compile the chamfer distance as indicated in the Readme ,the pytorch version I use is 0.3.0. The output of the chamfer distance on the toy example is like this.

cpu
(Variable containing:

Columns 0 to 7
6.1793 8.2053 11.5992 17.8038 14.3894 0.7531 6.0164 6.5923

Columns 8 to 15
31.9987 13.9602 15.7447 2.2181 13.0434 20.6023 2.0356 2.1271

Columns 16 to 23
4.4017 13.9608 9.8704 9.6264 16.3925 27.0252 9.5832 19.5379

Columns 24 to 31
6.4978 5.6396 13.3209 10.5200 1.8960 15.2886 29.5049 2.1821

Columns 32 to 39
14.0835 27.1476 12.0998 10.2464 3.7691 15.2096 19.5777 18.4997

Columns 40 to 47
14.2707 15.2700 12.0981 13.5516 7.8063 10.4152 6.5572 3.8471

Columns 48 to 49
22.2432 0.1973
[torch.FloatTensor of size 1x50]
, Variable containing:

Columns 0 to 7
2.1271 12.0998 3.8471 13.9608 19.5379 22.6752 12.0981 13.9602

Columns 8 to 15
10.4152 12.4652 6.5572 11.2524 13.1894 9.5832 9.8704 11.1269

Columns 16 to 23
16.3925 21.1371 8.2053 14.3894 3.6068 7.5756 0.1973 2.2181

Columns 24 to 31
14.2707 15.2096 10.5200 19.5777 18.4370 10.2464 9.6264 1.8960

Columns 32 to 39
13.5516 9.6820 3.7691 26.7427 4.2241 4.4017 27.0970 30.4567

Columns 40 to 47
23.1334 6.5923 6.0164 13.5451 0.7531 22.9326 6.7057 5.6396

Columns 48 to 49
2.0356 13.3209
[torch.FloatTensor of size 1x50]
)
Variable containing:
595.4069
[torch.FloatTensor of size 1]

(Variable containing:
(0 ,.,.) =
1.2849 4.3947 1.9372
-1.6723 -5.4728 0.2703
-0.2552 -2.4277 6.3591
3.0284 -7.8519 0.6256
5.4037 -2.5118 -4.6956
0.2393 -0.7758 -1.5341
-2.6297 4.1397 -0.1141
-3.7541 3.4075 0.8155
10.7046 -1.2437 3.4437
3.4121 -3.5196 5.6401
0.1679 5.7921 5.4223
1.1857 -2.6043 0.8272
0.5878 6.2924 3.4976
4.3331 -6.8907 4.0190
1.3072 -0.4982 2.4871
0.5679 2.8592 0.1029
-3.4618 -1.4108 1.9060
-3.3218 -0.4171 -6.6809
-0.9854 5.4946 2.8845
-1.3924 5.8655 -1.4705
-3.2039 4.4703 5.9432
9.5107 -3.5650 -2.2221
-2.3936 0.5873 5.6797
-6.7540 -4.9437 2.8451
4.1969 0.6256 2.8259
-2.3853 0.8255 -4.0233
-1.6369 -5.4689 4.5491
-0.8688 -0.5104 6.4082
0.0100 1.6627 -2.1953
-2.4424 4.2372 -6.1021
10.5281 -2.0631 1.7093
-2.8093 -0.1574 -0.9009
0.7212 -0.2210 -7.4676
-10.2857 1.2293 -1.1330
-6.6148 1.0285 -1.8938
3.0426 -1.4912 5.4318
3.7530 0.9296 -0.3569
0.7986 -5.5969 5.3736
-1.5220 6.9088 -5.3163
-3.7820 -1.6063 -7.5575
-0.3343 -6.8112 3.2525
7.0003 2.7021 -2.1852
0.6615 -5.7299 -3.8889
-6.8928 -2.5547 0.4113
-5.4322 1.3098 0.0337
5.9139 -2.0757 1.5420
-2.7526 -2.3126 3.6475
-1.4035 -0.4701 3.6328
3.7732 5.7402 6.4642
0.6029 -0.6186 0.2069
[torch.FloatTensor of size 1x50x3]
, None)
gpu
(Variable containing:

Columns 0 to 7
6.1793 8.2053 11.5992 17.8038 14.3894 0.7531 6.0164 6.5923

Columns 8 to 15
31.9987 13.9602 15.7447 2.2181 13.0434 20.6023 2.0356 2.1271

Columns 16 to 23
4.4017 13.9608 9.8704 9.6264 16.3925 27.0252 9.5832 19.5379

Columns 24 to 31
6.4978 5.6396 13.3209 10.5200 1.8960 15.2886 29.5049 2.1821

Columns 32 to 39
14.0835 27.1476 12.0998 10.2464 3.7691 15.2096 19.5777 18.4997

Columns 40 to 47
14.2707 15.2700 12.0981 13.5516 7.8063 10.4152 6.5572 3.8471

Columns 48 to 49
22.2432 0.1973
[torch.cuda.FloatTensor of size 1x50 (GPU 0)]
, Variable containing:

Columns 0 to 7
2.1271 12.0998 3.8471 13.9608 19.5379 22.6752 12.0981 13.9602

Columns 8 to 15
10.4152 12.4652 6.5572 11.2524 13.1894 9.5832 9.8704 11.1269

Columns 16 to 23
16.3925 21.1371 8.2053 14.3894 3.6068 7.5756 0.1973 2.2181

Columns 24 to 31
14.2707 15.2096 10.5200 19.5777 18.4370 10.2464 9.6264 1.8960

Columns 32 to 39
13.5516 9.6820 3.7691 26.7427 4.2241 4.4017 27.0970 30.4567

Columns 40 to 47
23.1334 6.5923 6.0164 13.5451 0.7531 22.9326 6.7057 5.6396

Columns 48 to 49
2.0356 13.3209
[torch.cuda.FloatTensor of size 1x50 (GPU 0)]
)
Variable containing:
595.4069
[torch.cuda.FloatTensor of size 1 (GPU 0)]

(Variable containing:
(0 ,.,.) =
1.2849 4.3947 1.9372
-1.6723 -5.4728 0.2703
-0.2552 -2.4277 6.3591
3.0284 -7.8519 0.6256
5.4037 -2.5118 -4.6956
0.2393 -0.7758 -1.5341
-2.6297 4.1397 -0.1141
-3.7541 3.4075 0.8155
10.7046 -1.2437 3.4437
3.4121 -3.5196 5.6401
0.1679 5.7921 5.4223
1.1857 -2.6043 0.8272
0.5878 6.2924 3.4976
4.3331 -6.8907 4.0190
1.3072 -0.4982 2.4871
0.5679 2.8592 0.1029
-3.4618 -1.4108 1.9060
-3.3218 -0.4171 -6.6809
-0.9854 5.4946 2.8845
-1.3924 5.8655 -1.4705
-3.2039 4.4703 5.9432
9.5107 -3.5650 -2.2221
-2.3936 0.5873 5.6797
-6.7540 -4.9437 2.8451
4.1969 0.6256 2.8259
-2.3853 0.8255 -4.0233
-1.6369 -5.4689 4.5491
-0.8688 -0.5104 6.4082
0.0100 1.6627 -2.1953
-2.4424 4.2372 -6.1021
10.5281 -2.0631 1.7093
-2.8093 -0.1574 -0.9009
0.7212 -0.2210 -7.4676
-10.2857 1.2293 -1.1330
-6.6148 1.0285 -1.8938
3.0426 -1.4912 5.4318
3.7530 0.9296 -0.3569
0.7986 -5.5969 5.3736
-1.5220 6.9088 -5.3163
-3.7820 -1.6063 -7.5575
-0.3343 -6.8112 3.2525
7.0003 2.7021 -2.1852
0.6615 -5.7299 -3.8889
-6.8928 -2.5547 0.4113
-5.4322 1.3098 0.0337
5.9139 -2.0757 1.5420
-2.7526 -2.3126 3.6475
-1.4035 -0.4701 3.6328
3.7732 5.7402 6.4642
0.6029 -0.6186 0.2069
[torch.cuda.FloatTensor of size 1x50x3 (GPU 0)]
, None)
stats:
('loss :', Variable containing:
595.4069
[torch.FloatTensor of size 1]
, Variable containing:
595.4069
[torch.cuda.FloatTensor of size 1 (GPU 0)]
)

from atlasnet.

Hi @AkonLau , what was the issue ?

from atlasnet.

Hi, it is an issue about the pytorch version.I used the pytorch version 0.1.12 instead of 0.3.1,and it works. It seems that the chamfer distance code also doesn't work on pytorch 0.3.1.

from atlasnet.

Ok, thanks a lot for reporting this, I'll add it to the README. I use the latest sources 0.4.0 with CUDA 9.0 and it works. below the learning curves you should get. Don't hesitate to reports issues, or things you found not obvious in the setup as you're among the first to use it.

from atlasnet.