Papermill hangs if kernel is killed about papermill HOT 5 CLOSED

I was thinking we want is to get the id of the kernel process and just monitor that the pid is still alive.

from papermill.

Can we add a thread that receives the kernels heartbeat and raises an exception when it stops receiving it?

An alternative is to have a timeout of N minutes of nothing happening. I don't like that option because you need to set it to (very) large numbers for notebooks doing "lots of stuff" and often I don't know what to set the timeout to. (For scikit-optimize we kept having this problem because travis nodes are a lot slower than laptops so tuning the timeout was tedious because you had to run it on travis...)

from papermill.

I definitely don't want a timeout as we should be able to have very long running jobs. Is there something on the nbconvert side or jupyter client side that could be catching more process level events or if we need to track the pid of the kernel process more directly?

from papermill.

jupyter/nbconvert#791 a discussion about advantages and disadvantages of having the default timeout in nbconvert. Might be interesting to observe what people say.

from papermill.

Solved in nbconvert 5.5 which is shipping tomorrow.

from papermill.