limaosen0 / as-gcn Goto Github PK

I'm really interested in the paper! I have trained this model about 100 epochs for several times. Howerver, I can't achieve the results described by the paper in NTU RGBD dataset.

1. I have tried to train the model by using the hyper-parameters given by the yaml file, the accuracy is only about 48%.
   

2. I noticed the hyper-parameters is different from the paper, so I changed them to be the same as the paper. Finally, the accuracy is about 80%.

3. I have also tried the older branch code committed by the author in May. The link is here: https://github.com/limaosen0/AS-GCN/tree/8efd973a8ecd37e22bb976de5cb49e1f7cb5724e
   But the accuracy is about 74%. I used the hyper-parameters given by the paper. The loss is about 0.38 after about 100 epochs， and it seems the loss will no longer decrease.
   

Does anyone achieve the results discribed by the paper? Can anyone tell me what's wrong with my experiments? Hope your reply! Thanks!

when I begin to PretrainAIM，it has a problem which is
File "main.py", line 4, in
from torchlight import import_class
ImportError: cannot import name 'import_class'

How to deal with it?

`class Model(nn.Module):
······
def forward(self, x, x_target, x_last, A_act, lamda_act):
print("A_act.shape = {}, lamda_act = {}".format(A_act.shape)

A_act.shape=2,2,25,25
What's the A_act?

Like the samples_with_missing_skeletons.txt you provided for NTU-RGB+D,I would appreciate it if you could show me the same way to get the file for NTU-RGB+D120.

I did the warmup pretraining for 9 epochs (instead of your 10 epochs) and that worked okay, and wanted to continue with the train part but hit that error then on the 11th epoch (thus can train for the 10th epoch, but a magical error then happens on the 11th?, and can't permute the x_last)

[08.13.19|01:43:40] Training epoch: 9
AS-GCN/net/utils/adj_learn.py:11: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.
  soft_max_1d = F.softmax(trans_input)
[08.13.19|01:43:41] 	Iter 0 Done. | loss2: 1961.2838 | loss_nll: 1938.3290 | loss_kl: 22.9549 | lr: 0.000500
[08.13.19|01:44:21] 	Iter 100 Done. | loss2: 1593.5586 | loss_nll: 1572.3160 | loss_kl: 21.2426 | lr: 0.000500
.....
[08.13.19|02:33:32] 	Iter 7400 Done. | loss2: 1933.1755 | loss_nll: 1911.2896 | loss_kl: 21.8860 | lr: 0.000500
[08.13.19|02:34:12] 	Iter 7500 Done. | loss2: 1391.6711 | loss_nll: 1369.8772 | loss_kl: 21.7940 | lr: 0.000500
[08.13.19|02:34:17] 	mean_loss2: 1974.6090346069418
[08.13.19|02:34:17] 	mean_loss_nll: 1951.943839369832
[08.13.19|02:34:17] 	mean_loss_kl: 22.665194850461106
[08.13.19|02:34:17] Time consumption:
[08.13.19|02:34:17] Done.
[08.13.19|02:34:17] The model has been saved as ./work_dir/recognition/kinetics/AS_GCN/max_hop_4/lamda_05/epoch9_model1.pt.
[08.13.19|02:34:17] The model has been saved as ./work_dir/recognition/kinetics/AS_GCN/max_hop_4/lamda_05/epoch9_model2.pt.
[08.13.19|02:34:17] Eval epoch: 9
[08.13.19|02:36:22] 	mean_loss2: 2030.5040628313056
[08.13.19|02:36:22] 	mean_loss_nll: 2008.193604698859
[08.13.19|02:36:22] 	mean_loss_kl: 22.310456798226845
[08.13.19|02:36:22] Done.
[08.13.19|02:36:22] Training epoch: 10

Traceback (most recent call last):
  File "main.py", line 30, in <module>
    p.start()
  File "AS-GCN/processor/processor.py", line 111, in start
    self.train(training_A=False)
  File "AS-GCN/processor/recognition.py", line 161, in train
    x_class, pred, target = self.model1(data, target_data, data_last, A_batch, self.arg.lamda_act)
  File "/home/petteri/anaconda3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 547, in __call__
    result = self.forward(*input, **kwargs)
  File "AS-GCN/net/as_gcn.py", line 59, in forward
    x_last = x_last.permute(0,4,1,2,3).contiguous().view(-1,3,1,25)
RuntimeError: shape '[-1, 3, 1, 25]' is invalid for input of size 3456

And I assume that the error is being propagated from the data generation script for which I used the npy generation code from 2s-AGCN implementation: https://github.com/lshiwjx/2s-AGCN/blob/master/data_gen/kinetics_gendata.py

As when looking at the size, is it so that the line is defined statically and the last 25 refers to the number of joints in the dataset, and when I use the kinetics instead of the NTU-RGBD this should be conditionally set as you do here for the openpose?

x_last = x_last.permute(0,4,1,2,3).contiguous().view(-1,3,1,25)

32 3 290 18 2 # (N, C, T, V, M)
x_last:  torch.Size([32, 3, 1, 18, 2])
x_recon:  torch.Size([32, 3, 290, 18])
x1:  torch.Size([32, 3, 290, 18, 2])
x2:  torch.Size([32, 2, 18, 3, 290])
x3:  torch.Size([64, 54, 290])

printed like this

def forward(self, x, x_target, x_last, A_act, lamda_act):
     N, C, T, V, M = x.size()
     print(N, C, T, V, M)
     print('x_last: ', x_last.shape)
     x_recon = x[:,:,:,:,0]                                  # [2N, 3, 300, 25]
     print('x_recon: ', x_recon.shape)
     print('x1: ', x.shape)
     x = x.permute(0, 4, 3, 1, 2).contiguous()               # [N, 2, 25, 3, 300]
     print('x2: ', x.shape)
     x = x.view(N * M, V * C, T)                             # [2N, 75, 300]
     print('x3: ', x.shape)
     x_last = x_last.permute(0,4,1,2,3).contiguous().view(-1,3,1,25)
     print('x_last: ', x_last.shape)

File Not Found Error: [Errno 2] No such file or directory: './work_dir/recognition/ntu-xsub/AS_GCN/max_hop_4/lamda_05/epoch99_model1.pt'

I find that in ST-GCN there is a pretrained model provided by autor, where is the pretrained model for this repo?

While running the code, I ran into another issue. I was using this instruction:

Train: python main.py recognition -c config/as_gcn/ntu-xsub/train.yaml

But I got :

ModuleNotFoundError: No module named 'feeder'

I assume that feeder is also a folder containing these functions to act as data loaders for the training. It will be great if you could update the code with those libraries.

Hi,

Do you happen to have the kinetics config available to share so that your results are easier to reproduce as your repo seemed to contain only the NTU-RGBD configs?

Thanks

I think your work is wonderful, and I want to use your models to complete some works. Can you provide the trained models?

https://github.com/1suancaiyu/AS-GCN

I am confused with this line of code. If hard is set as True even for training, how can the gradient flow back through AIM module in backprop?

how can I solve it?

I have finished the torchlight installed ,but it remains the import error:
Processing dependencies for torchlight==1.0
Finished processing dependencies for torchlight==1.0

ImportError: cannot import name 'import_class' from 'torchlight'

I have completed the related work of ST-GCN, and I found that your data processing code is similar to their code, so can I use the ST-GCN dataset on your code directly?

ok,I already understand

Firstly, thanks a lot for making the code public. I am trying to train AS-GCN on the cross-setup setting as described in NTU RGB D 120. For performing this experiment, I am doing the following steps:

1. Making changes in the preprocessing file to create new data with 120 classes and even setups for training and odd setups for testing.

2. I make the changes in the yaml config file and following the same procedure as given for the NTU RGB 60.

I am getting decent accuracy for the Cross-Subject protocol but really low on Cross setup.
What can be the reason for this?

i have question

what means data_downsample, target_data, data_last ?

dome

Thanks for your code.

I have already pretrained AIM first and it's good. When I trained the main pipleline by

python main.py recognition -c config/as_gcn/ntu-xsub/train.yaml

I got this error:

Traceback (most recent call last):
  File "main.py", line 24, in <module>
    p.start()
  File "/home/lenovo/code/tang/AS-GCN/processor/processor.py", line 111, in start
    self.train(training_A=False)
  File "/home/lenovo/code/tang/AS-GCN/processor/recognition.py", line 167, in train
    loss1.backward()
  File "/home/lenovo/.conda/envs/tang0.4.0/lib/python3.6/site-packages/torch/tensor.py", line 102, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph)
  File "/home/lenovo/.conda/envs/tang0.4.0/lib/python3.6/site-packages/torch/autograd/__init__.py", line 90, in backward
    allow_unreachable=True)  # allow_unreachable flag
RuntimeError: derivative for to_dense is not implemented

I don't know how to fix the error above. I would appreciate it if you could help me.

i got this error when i run: python main.py recognition -c config/as_gcn/ntu-xsub/train.yaml

I'm using the default conf train.yaml (4 gpus)

[09.25.19|12:58:47] Training epoch: 10
/scratch/oussema/AS-GCN/processor/io.py:35: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
default_arg = yaml.load(f)
/pytorch/aten/src/THCUNN/ClassNLLCriterion.cu:105: void cunn_ClassNLLCriterion_updateOutput_kernel(Dtype *, Dtype *, Dtype *, long *, Dtype *, int, int, int, int, long) [with Dtype = float, Acctype = float]: block: [0,0,0], thread: [7,0,0] Assertion t >= 0 && t < n_classes failed.
/pytorch/aten/src/THCUNN/ClassNLLCriterion.cu:105: void cunn_ClassNLLCriterion_updateOutput_kernel(Dtype *, Dtype *, Dtype *, long *, Dtype *, int, int, int, int, long) [with Dtype = float, Acctype = float]: block: [0,0,0], thread: [8,0,0] Assertion t >= 0 && t < n_classes failed.
/pytorch/aten/src/THCUNN/ClassNLLCriterion.cu:105: void cunn_ClassNLLCriterion_updateOutput_kernel(Dtype *, Dtype *, Dtype *, long *, Dtype *, int, int, int, int, long) [with Dtype = float, Acctype = float]: block: [0,0,0], thread: [17,0,0] Assertion t >= 0 && t < n_classes failed.
/scratch/oussema/AS-GCN/net/utils/adj_learn.py:11: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.
soft_max_1d = F.softmax(trans_input)
Traceback (most recent call last):
File "main.py", line 24, in
p.start()
File "/scratch/oussema/AS-GCN/processor/processor.py", line 111, in start
self.train(training_A=False)
File "/scratch/oussema/AS-GCN/processor/recognition.py", line 163, in train
loss_recon = self.loss_pred(pred, target)
File "/home/oussema/as-gcn/lib/python3.6/site-packages/torch/nn/modules/module.py", line 547, in call
result = self.forward(*input, **kwargs)
File "/home/oussema/as-gcn/lib/python3.6/site-packages/torch/nn/modules/loss.py", line 431, in forward
return F.mse_loss(input, target, reduction=self.reduction)
File "/home/oussema/as-gcn/lib/python3.6/site-packages/torch/nn/functional.py", line 2190, in mse_loss
ret = torch._C._nn.mse_loss(expanded_input, expanded_target, _Reduction.get_enum(reduction))
RuntimeError: after reduction step 1: device-side assert triggered

hello, i have question about as-gcn algorithm code.
First, in which file is the code to process 2D data in the AS-GCN algorthm?
Second, in which file does the AS-GCN algorithm process the kinetic data file?

thank you :)

File "main.py", line 4, in
from torchlight import import_class
ImportError: cannot import name 'import_class'

File "/AS-GCN/io.py", line 183, in import_class
import(mod_str)
ModuleNotFoundError: No module named 'processor.demo'

Since I didn't find the code you used to package the data, I used the ntu rgb-d dataset directly when training st-gcn, but this input doesn't seem to fit your model. Can you upload this part of the code or your packaged data set?

没找到你用来打包数据的代码，我直接使用了在训练st-gcn时打包ntu rgb-d数据集，但是这个输入似乎不适合你的模型。你能上传这部分的代码或者你打包好的数据集吗？

When I run the main function, I encountered this problem, how can I solve it? Thank you.

Traceback (most recent call last):
File "D:/AS-GCN-master/main.py", line 22, in
Processor = processors[arg.processor]
KeyError: None

When I was running this coding'' python main.py recognition -c config/as_gcn/ntu-xsub/train_aim.yaml'',there was a error''main.py: error: unrecognized arguments: recognition -c config/as_gcn/ntu-xsub/train_aim.yaml''

when I run a commad "python ntu_gen_preprocess.py
xsub train".It appears the problem :Traceback (most recent call last):
File "ntu_gen_preprocess.py", line 143, in
gendata(arg.data_path, out_path, arg.ignored_sample_path, benchmark=b, part=sn)
TypeError: gendata() got an unexpected keyword argument 'part'

Hello, I will really appreciate if you upload some pretrained models.

1.In tools.util.nut_gendata.py

line 12 :from utils.ntu_read_skeleton import read_xyz
I can't import utils and read_xyz
and when i try ro run it ,it comes to this

Traceback (most recent call last):
File "C:/Users/little bai/Desktop/work work/st-gcn-master/tools/ntu_gendata.py", line 125, in
part=p)
File "C:/Users/little bai/Desktop/work work/st-gcn-master/tools/ntu_gendata.py", line 45, in gendata
with open(ignored_sample_path, 'r') as f:
FileNotFoundError: [Errno 2] No such file or directory: 'resource/NTU-RGB-D/samples_with_missing_skeletons.txt'

2.In main.py
When i try to run main.py ,it goes wrong

Traceback (most recent call last):
File "C:/Users/little bai/Desktop/work work/st-gcn-master/main.py", line 32, in
Processor = processors[arg.processor]
KeyError: None

3.How should i dispose the dataset?
where should i place the NTURGB+D files?
and what should i do to manage the dataset ?

4.I am a newfish , could i get some coures in detial or would you give me some suggests?

In the end : so much thanks for your help,and i will feel grateful if you can help me for running the st-gcn models!!!!!!

Hi @limaosen0 , thanks for your great work!
Could you please introduce your 'preprocess.py', confusing about the function 'pre_normalization'? Could you give me some advice?

Thanks very much!

There were two problems when I started to train the main pipleline of AS-GCN ,as shown in the title.
In addition, I found that this project uses four GPUs to perform this program, but my computer only has one GPU. Can I run this project? Can anyone tell me what's wrong with my experiments? Hope your reply! Thanks!

Hello, I would like to ask how the final skeleton diagram of the feature in the paper was drawn.

could you mention which part of A-act in your code? I didn't find it.

Hello, I am very interested in your thesis, but it is difficult to download the dataset. If you can see this message, can you pass it to me？ thank you.

你好，请问这个问题有解决办法吗？找了好久没找出来

Can you explain the AIM's loss function? I can understand it. Thank you vert much！

谢谢大牛

            rel_rec = self.rel_rec.cuda(gpu_id)
	rel_send = self.rel_send.cuda(gpu_id)

	self.logits = self.encoder(x, rel_rec, rel_send) 

The dim of rel_rec and rel_send is 【600，25】,but I can not understand what they mean about human skeleton. AIM network is a little hard to understand. I would appreciate it if you could elaborate on your ideas. Thanks!!

Traceback (most recent call last):
File "./data_gen/ntu_gen_preprocess.py", line 7, in
from data_gen.preprocess import pre_normalization
ModuleNotFoundError: No module named 'data_gen'

First of all, thanks a lot for providing this code. It is a great help for beginners like me.

I am trying to train AS-GCN on NTU RGB-D 120 ( the new dataset). I convert the data into the format: [N x C x max_frame_len x no_joints x no_subjects], N = total number of samples,
C = total number of channels[3], max_frame_len = 300, no_joints= 25, no_subjects = 2 and save it in a .npy format.

I am then setting up the config.yaml file and trying to train using:

Train: python main.py recognition -c config/as_gcn/ntu-xsub/train.yaml

but I keep getting this ModuleNotFoundError: No module named 'processor.demo'.

In the Processor folder, I couldn't find any file with the name demo either. It will be great if you could provide it if it is necessary for the training of the program.