perf: notebook serialization for large notebooks blocks cell execution about vscode HOT 3 OPEN

@arman258 I created another issue for that since it doesn't have anything to do with serializing large notebooks.

from vscode.

I played around with an incremental serialization that would allow for more responsiveness (draft PR), basically serializing the code cells one at a time and checking for cancellation in between and the results were not very promising - serialization takes around 4x as long and still has some pretty significant hiccups in execution time.

@rebornix - do you have any ideas for improvement here? Perhaps something like a dedicated worker thread on the EH?

from vscode.

@amunger coming from #211154, I think copilot has something to do with this issue. Take a look at the videos below; I just started working on another .ipynb file, not even 2 cells with about 20-30 lines of codes, the slowness started. You can clearly see the copilot trying to do something and lagging the jupyter to show the output.

I also face the same issue with another AI code completion called Codeium. The behavior is the same. Disabling them seems to handle the jupyter files better, but as the file starts to get complicated, it gets slow again.

ms-toolsai.jupyter-unresponsive.cpuprofile.txt

from vscode.