cmu-perceptual-computing-lab / ssp Goto Github PK

Hello!

I noticed that in addition to the startFrame in the pickled dictionary for each group, there is an entry for the start frame in the data dictionary for each subject as well. These don't match, so I am assuming that the start frame in the subject dictionary represents the first frame where they are visible in the scene. However, the data array for each subject has the same shape.

So I am wondering if the data is represented starting at the start frame for the haggling game, or the start frame mentioned in the data dictionary. Here is a running example using seq 170221_haggling_b1_group0:

In [23]: with open("170221_haggling_b1_group0.pkl", "rb") as f:
        ...:     data = pickle.load(f, encoding="latin1")

In [24]: data["startFrame"]
Out[24]: 1223

In [25]: data["subjects"][0].keys()
Out[25]: dict_keys(['joints19', 'bodyNormal', 'faceNormal', 'humanId', 'scores', 'startFrame', 'bValid'])

In [26]: data["subjects"][0]["startFrame"]
Out[26]: 1167

In [27]: data["subjects"][1]["startFrame"]
Out[27]: 1078

In [28]: data["subjects"][2]["startFrame"]
Out[28]: 1125

In [29]: data["subjects"][0]["bodyNormal"].shape
Out[29]: (3, 1832)

In [30]: data["subjects"][0]["joints19"].shape
Out[30]: (57, 1832)

In [31]: data["subjects"][1]["bodyNormal"].shape
Out[31]: (3, 1832)

In [32]: 1223 + 1832
Out[32]: 3055

Downloading the video for the sequence and extracting the individual frames, it seems like the participants start leaving shortly after frame 3055, leading me to believe that the data in the subjects dictionary is for the frames explicitly involving the game, and the startFrame in the subject dictionary is just auxiliary information not pertaining to the data.

On the same note of auxiliary data, how should one interpret bValid? It's a single boolean while the scores are provided for each frame, so I'm not sure how the two pieces combine.

Can someone confirm/clarify these please? Thanks!