Runtime Performance about pvnet HOT 8 CLOSED

Hi PedroCastro,
The former testing code didn't use synchronize, which caused incorrect time records for each stage.
I add torch.cuda.synchronize() and find the forward pass indeed costs about 27 ms and the RANSAC costs about 14ms.
Thank you for pointing out the mistakes!

from pvnet.

How about testing several times and taking the average time?
Your RANSAC and PnP performance is similar to us, but the forward pass has a large gap.
Our running time does not include the metric measuring.
The reason our method is real-time is that our backbone is Resnet-18.

from pvnet.

Thanks for the answer!

Well, I am already taking the average among all test images for the Ape object. Do you mean taking the avg of multiple objects? That shouldn't change the inference speed so I am not sure what I can measure differently.

I agree that Resnet-18 is quite fast. But for example, this pre-print which also uses Resnet-18 with a much smaller input (320x240) claims their forward propagation takes 11ms, on a Titan X.

Am I measuring a different call than what you guys did? How did you measure it?

Thanks again for the conversation!

from pvnet.

Their architecture has three decoders.
I will ask my partner to double check our forward pass time.

from pvnet.

Does your forward pass include the data loading?

from pvnet.

Oh @pengsida, you are right! But even if you divide the full time by 3 (which would divide the time on the encoder part as well) it would match your speed with half the resolution.

And no, it doesnt take the data loading into account! I synchronize cuda calls before and after the forward pass.
My code is like this:

torch.cuda.synchronize()

start.record()

seg_pred, vertex_pred, loss_seg, loss_vertex, precision, recall = net(image, mask, vertex, vertex_weights)

torch.cuda.synchronize()

end.record()

time_accumulator += [start.elapsed_time(end)]

and then I avg the time at the end.

I think your method does run at 25fps but the way your time is discretized in the paper might not be right.

from pvnet.

Probably.
A double check is needed.
It is indeed hard to directly measure the cuda running time.
Whatever, our approach is real-time in our use.

from pvnet.

Yes, good job and good paper ;)

Let me know what you find!

from pvnet.