Clarification regarding the label "crossing" in the Annotation Attributes about pie HOT 4 CLOSED

Hi @Ruthvik9,
Thank you for your interest in our work. Regarding your questions:

1. Per-frame labels may indicate that the pedestrian started to cross, i.e., stepped onto the road. But the crossing label for the entire track is set to 1 only if the pedestrian actually crossed the road in front of the vehicle (traversed its width or crossed in its path if the vehicle is turning). In other words, per-track labels for crossing reflect the final outcome of the pedestrian's actions relative to the ego-vehicle. Even if the pedestrian eventually crossed after the ego-vehicle went past them, we assign the "non-crossing" label to the track because they did not cross in front of the ego-vehicle.

2. Yes, we used the per-track labels as the ground truth for the benchmark and all samples were collected before crossing. Otherwise, if a person is mid-crossing, it is more of a detection rather than prediction problem. In that paper, we did not analyze the post-crossing activity.

from pie.

Thank you so much for the prompt response!

After going through your "Benchmark for evaluating Pedestrian Action Prediction" work, I had another doubt regarding these annotations.
In that paper, you mention the following -


i.e., "The objective is to predict whether the pedestrian will start crossing the street at some time t given the observation
of length m. We define the event at the time the pedestrian starts to cross or the last frame the pedestrian is observable in case no crossing takes place."

i) Is it "whether the pedestrian will start crossing the street" as present in the paper or something like "whether the pedestrian will cross the street in front of the ego-vehicle, predicted t frames before the event"(with the event being defined as the time the pedestrian
starts to cross, not necessarily in front of the ego-vehicle)? Because as per your response, I believe it should be the latter.
ii) In case no crossing takes place, is it the last observable frame that's defined as the event or the last observed frame - 3 as you've defined in this work?

Many thanks!

from pie.

Yes, when predicting the crossing action, we care whether the pedestrian will eventually end up crossing in front of the car. When the pedestrian is not crossing, we consider the event as the last observed frame - 3 (in the last few frames of the track, only a tiny portion of the pedestrian may still be visible, e.g. a foot).

from pie.