"standard_error" on submission about habitat-challenge HOT 6 CLOSED

We're working on all of these points.

from habitat-challenge.

Hi Devendra, that's not expected behavior of EvalAI and will be fixed. We will try our best to provide you error logs.

from habitat-challenge.

Hi @devendrachaplot , your submission to the minival phase is timing out as it isn't able to run through all the episodes in 30 mins.
(We will fix the error message at our end to be more meaningful.)

Please close this issue if this resolves your query.

from habitat-challenge.

It is easy to bring your run-time down? We can bump the time-out limit but it will reduce your iteration cycles when you get to test-std (which is 100x larger than minival).

from habitat-challenge.

It will be great if the time-out limit can be increased! The local evaluation takes around 16 mins on my machine. Is there a way to see the output of the model? Our submission prints FPS. I am trying to figure out the ratio of FPS from my machine to remote evaluation.

from habitat-challenge.

Thanks, the time limit issue was mostly because of inconsistent docker base image (https://github.com/facebookresearch/habitat-challenge/issues/33#issuecomment-629741382). Fixing the base docker image reduced the runtime by around 50%.

Also, just for your reference, the remote eval (with the fixed docker base image) is now taking around 2x time as compared to my local evaluation on with 1080 Ti GPU (14 mins vs 7 mins).

from habitat-challenge.