Question about training the fitting stage about multimodal-future-prediction HOT 3 CLOSED

Hi,

The first thing I encounter is what you have at line 80:
you need to use the bounded_log_sigmas instead of sigmas, so you need to change the second argument from output[1] to output[3]

Moreover, you need to ensure that when training the fitting network that the gradient is not passing backward to the sampling network. You can ensure that by adding stop_gradient on input_2 in net.py (make_graph() at line 43)

Note that during training the fitting, the NLL loss usually does not decrease much but it should not be always constant. Maybe you can share the fitting loss plot.

Depending on your application, you need to check the quality of the hypotheses from the sampling network to see if they are good. If not, the fitting network won't be able to fit a mixture model to bad hypotheses.

Hope this helps,

from multimodal-future-prediction.

Thanks for your findings, the change of output[1] to output[3] solved the constant fitting loss problem. I also added: input_2 = tf.stop_gradient(input_2) before the 'net2' layers, but there is not much difference on the loss compared to not using the stop_gradient. However, the errors are still there, I will train it with more data and epochs to see if these errors affect the performance.

I have not visualized the result of the hypothesis but will do it later. My training loss of the hypothesis is around 11 after 500 epochs of training on 10k training pairs.

from multimodal-future-prediction.

I think this can be closed for now. Please feel free to reopen it if you have encounter other problems.

from multimodal-future-prediction.