MTGNN only captures the first two steps and then fails about bigscity-libcity HOT 8 CLOSED

I guess maybe you don't train this model for 100 epochs. MTGNN propose a curriculum learning strategy to train multi-step model, i.e. the task_level gradually +1 for every step_size. Since we predict 12 steps, the model must iterate enough rounds to increase the task_level to 12, so that all steps are fully trained. Related code is here MTGNN.

If you transpose the origin shape (batch_size, input_window, num_nodes, feature_dim) into (batch_size, feature_dim, num_nodes, input_window), in our code, the max task_level will equal to feature_dim=1. Therefore, it is not necessary to train too many rounds to make the model fully trained, so the results appear to be normal.

However, this is inconsistent with the paper authors' approach. In the original author's code, the reason why transpose(1,3) is executed is because it performs transpose(1,3) on the predicted target trainy, and in order to make the predicted value match the shape of the true value, so he needs to perform a shape change. And the shape change actually does not affect the output of inverse_transform and transform of scaler. The original code is here Origin code.

In our code, no changes are made to the shape of trainy, and therefore no changes are needed to the shape of the model's output y_pred. The reason for the situation you posted is caused by task_level, just increase the training epoch or reduce the batch_size to reproduce the original performance.

from bigscity-libcity.

Bugs founded:

In MTGNN, The tensor before feeding into inverse_transform should be transposed as (batch_size, feature_dim, num_nodes, input_window).
See the code provided by the original author here:
https://github.com/nnzhan/MTGNN/blob/f811746fa7022ebf336f9ecd2434af5f365ecbf6/trainer.py#L48

That is, the original code

The performance now looks fine after applying those changes:

from bigscity-libcity.

Also to add, the author's code also supports task_level, the code is here. But because of the execution of transpose(1,3), his task_level is in the fourth dimension predict[:, :, :, :self.task_level], while we are in the second dimension y_predicted[:, :self.task_level, :, :], which means that even if the original author's code is used, it is not possible to train the later time steps without training enough rounds.

Later I will add a log of my training process to you. If there are still questions, please continue to leave your comments. Feel free to give us a STAR if our code is helpful, thanks!

from bigscity-libcity.

from bigscity-libcity.

86194-MTGNN-METR_LA-May-22-2022_11-58-10.log

from bigscity-libcity.

You can mainly focus on the increasing task_level records in the log.

2022-05-22 11:58:47,795 - INFO - Training: task_level increase from 0 to 1
...
2022-05-22 12:44:58,146 - INFO - Training: task_level increase from 11 to 12

from bigscity-libcity.

Thanks for your patient response!

Yes, you are right, I only train 30 epochs and that is the reason why the first two steps are captured.

I totally miss the curriculum learning part in the MTGNN, and thanks for reminding me that! You can close this issue now. Apologize for my misleading message.

Again, a very nice library and thank you for your time to build it! I have starred it, and hope it would become more popular!

from bigscity-libcity.

Thank you!

from bigscity-libcity.