aras62 / piepredict Goto Github PK

Hello, great work @ykotseruba and team!
I was trying to use the piepredict model on custom dataset to predict pedestrian intention,but I am not sure how did you generate the annotation_attributes.zip (under pi dataset),using cvat tool.
I was able to generate the annotations using cvat tool ,but I am wondering how to get annotation_attributes.
Any suggestion is greatly appreciated!
Thanks.

Hi,

Many thanks for your nice work.

When I tried to run test using only seq1, I have memory error. I am wondering how much memory actually needed to run this model? Could you share some ideas? I have 16G memory, it seems that it is not enough..

Best wishes,
Xingchen

Thank you for your fantastic work. I'm really enjoying your papers:

• PIE: A Large-Scale Dataset and Models for Pedestrian Intention Estimation and Trajectory Prediction
• Pedestrian Action Anticipation using Contextual Feature Fusion in Stacked RNNs

These two approaches solve different problems: trajectory prediction and action anticipation. I have some concerns/ideas I'd like to share:

1. Intention Prediction/Classification shows a F1 score higher than Action Classification in your experiments (0.87 vs. 0.72). It's quite a big difference. Do you think is a matter of the labels balance for both cases?
2. Do you think Action Classification could also benefit from previous Intention Classification?
3. Intention Classification is only present in your dataset, so training your models over other datasets is not possible. Do you think that trajectory prediction could also be refined with Action Classification? I mean, replacing the Intention Classification model by an Action Classification model
4. I noticed in your models that you first train the intention prediction model and then, the trajectory predicition one. Could it be interesting to set some common parts to both networks to train a multi-task learning fashion model (both models at the same time)?

Thank you very much again for your work, it is very inspiring!

hi sir, when I down the PIE , the /PIE/ has 3 zip (annotations.zip, annotations_attributes.zip and annotations_vehicle.zip). however , according to github in /PIEPredict/PIE_dataset , it need only annotations folder. I need copy set01-06 from annotations_attributes and annotations_vehicle , is true?

Bonjour,

J'utilise actuellement le script pour l'intention du piéton, cependant j'ai lancé à plusieurs reprises l'entrainement et le test mais je trouve une précision d'environ 28%. Est ce normale ?
Je vous remercie par avance pour votre réponse.

hi sir, I`m so sorry to bother you.
when i input "python train_test.py 1", it will occur :

Using TensorFlow backend.
Traceback (most recent call last):
File "train_test.py", line 182, in
main(train_test=train_test)
File "train_test.py", line 175, in main
intent_model_path = train_intent(train_test=train_test)
File "train_test.py", line 135, in train_intent
imdb = PIE(data_path=os.environ.copy()['PIE_PATH'])
File "/home/guofengpeng/my/pie/PIEPredict/pie_data.py", line 57, in init
'pie path does not exist: {}'.format(self._pie_path)
AssertionError: pie path does not exist: /path/to/PIE/data/root

but i cannot find where write "/path/to/PIE/data/root".

hi，sir. I want to make myself dataset. Could you tell me how you make the annotations ? I have know how to use cvat to annotate the video clips.

Holle，sir, thank your great job!
I have train the model, but I want to see the effect. I want to take a video ( or use a video from the datasets) , then input the model. At last I hope get a video annotated by the model. Please teach me how to use , thank.

Hi there,

I'm playing around with your repo, which is very well written actually. I'm trying to predict pedestrian intention (performing inference) in my own sequences, so I'm not using the pie_data.py... To check my pipe-line, I'm firstly predicting on the extraced images from the PIE_clips but I'm getting weird results (a prediction over 0.9 for every sequence)... I'm following your code for the pre-processing stages (jitter, squarify, crop and im_pad, vgg16.preprocess) so I think the problem shouldn't be there... (I'm assuming images are in RGB format, not BGR)

My problems might be with the bounding boxes:

1. Which bboxes/locations should be fed? original locations or expanded locations (with the 2x factor for local context)?
2. How the bounding box normalization is done?, I'm proceeding:

bbox[0]/=image_w
bbox[1]/=image_h
bbox[2]/=image_w
bbox[3]/=image_h

where bbox =[x_0, y_0, x_1, y_1] the top-left and botton-right corners of the bbox.

I could perform a pull request with the inferece code, which might be of great help (if it works) for performing easy inference for a given sequence of images

Thank you very much!

Hi there!

First of all thanks for your great work!
The code you provided is very easy to read and use, that's awesome!

After working with your code, I have some questions regarding the intention data that you use when testing the final model (test_final method in pie_predict.py).

If I understand correctly, you do not infer the intention values online for corresponding observation data, but instead load the intention results from the test phase of the intention module (pie_int).
When loading the intention results in the test stage of pie_traj, you follow two methods, if there is no intention data for a specific test sample.

1. If there are previously observed intention values for a ped_id, you use the average result as the intention value for that test sample. What is the reason for this procedure (I could not find it in the paper)?

2. Otherwise you use an intention value of 0.5 for the test sample. Since an intention value of 0.5 is only present for samples, that start before the defined exp_start_point, an intention value of 0.5 indicates a "flag" that the pedestrian will not cross the street in the near future. The model could use this information in the test stage. What do you think about this issue?

Apart from that I have one more question:
3. Why do you not infer the intention values for all test samples instead of loading the inferred results from the test stage of pie_int?

Thanks in advance
Phillip

I am using set01/video_0002 to run the train_test.py 2 (after changing the the pie_data.py file / line 92). Howver, I am encountering an issue for running the code as following :
"FileNotFoundError: [Errno 2] No such file or directory: '/home/marouene/PIEPredict/PIE_dataset/images/set01/video_0002/05051.png"
That being said, even for testing would the concerned video images shall be extracted !!
Many hanks for your quick reply

hi sir, I disturb again, sorry.
when I use 'python train_test.py 2', (the input is set03), I hope I can use opencv to display the trajectory. the output is like this image from ‘https://github.com/YapingZ/Pedestrian-behavior-trajectory-prediction’.
So, how can I get the value of 'x' and 'y' from output data?

Hi,

where is the config file to change extracting frames from true to false?

Thanks.

I have read the code carefully. How great a job !
today I make the result visualize , showing as follow.

Hi,

Thank you much for such a great work !. I am trying to understand how you de-normalize bounding box coordinates before evaluating the performance.

I understood that you normalized it by subtracting the box's coordinate in the first frame, but when I see the code below:

       perf = {}
        performance = np.square(test_results - box_data['pred_target'])
        perf['mse-15'] = performance[:, 0:15, :].mean(axis=None)
        perf['mse-30'] = performance[:, 0:30, :].mean(axis=None)  # 15:30
        perf['mse-45'] = performance.mean(axis=None)
        perf['mse-last'] = performance[:, -1, :].mean(axis=None)

in

You dont covert it back to actual pixel coordinates before calculating MSE for bounding box. I see that you do this denormalization for the centers before calculating C-MSE, which is correct to my thought, but this does not apply to bounding box coordinates ?

Could you please shed a light to this ? I believe that I am missing something here.

Thank you very much,

I have a question please.
It is logical that the tracks might have different length appearance in the video.
Is there a phase of pre-processing that you make them in the same size of length(which I didn't found it in function get_data) or the model deal with it automatically ?

Thank you

First of all thanks for your great work!
The code you provided is very easy to read and use, that's awesome!
I've been encoutring a problem while trying to train the train_test.py for test or train. After all the VGG16 extraction when it reachs 99% it gives me this error " Allocation of 9707550720 exceeds 10% of free system memory. " while I am working on a Cloud Service with 36Gb RAM