Chrome process isn't cleaned up when server is stopped about chromic_pdf HOT 3 CLOSED

@andreasknoepfle https://stackoverflow.com/a/39772976

from chromic_pdf.

More control to the user of the library is a good first step I believe, so on_demand mode sounds great to me. I wouldn't do any fancy things on Code.ensure_loaded?(IEx), since I would leave this up to the user to do 🤷.

from chromic_pdf.

@andreasknoepfle

research so far

• As outlined in the SO article, unix standard way of handling this is sending EOD which Chrome should handle. It does not. Also it should be possible for Chrome to determine that the remote debugging pipe is broken (after the BEAM is shut down), but it apparently doesn't care. Chrome seems really comfortable with just sticking around for a while.
• porcelain, which is a library for managing external processes, ships with an external binary called goon which serves as a "proxy" in between the BEAM and the external process and is capable of cleaning up when the BEAM dies. I think adopting porcelain would solve our issues, but at the cost of introducing another dependency plus this magic goon binary in the images. I don't think I'm ready for this.
• OTP 20 added erl_signal_server & friends, a way to handle unix signals in Erlang using gen_event.
  • Here's a fantastic article about how to make use of this to do a graceful service shutdown on Kubernetes (sends SIGTERM), draining HTTP request pipes before closing pods. Excellent read.
  • This is great and in fact I was able to capture SIGTERM, SIGABRT, and SIGQUIT with it and run some code. However it doesn't help us in this situation because ...
• When you hit Ctrl+C in Elixir or Erlang (from any BEAM-connected-to-console process really, doesn't have to be a shell), it drops you into the "break" menu. From here, hitting Ctrl+C again or using the "(a)bort" option executes erts_exit directly. I have to admit that I wasn't able to find the definition of erts_exit in the Erlang sources, but it is used all over the place and seems to be merely a wrapper for exit(2). Point is, I don't think there is any way of intercepting erts_exit and running any more clean-up code.

conclusion

• Cleaning up the Chrome process after the BEAM has been terminated with Ctrl+C does not seem to be (easily/without goon) possible.
• We could handle SIGTERM to ensure we clean-up properly on k8s, but I personally don't think this is the responsibility of this library (the host app should trap signals and shutdown its supervision tree, closing Chrome in the process). Besides, leaving a Chrome process running in a to-be-terminated k8s pod is likely not really a problem at all 😃 .
• Going down this path hasn't shown to be very fruitful, so IMO we should explore alternatives.

problem recap

• Terminating an elixir app with running ChromicPDF with Ctrl+C doesn't clean up Chrome properly.
• Did some more experiments with the tests
  • ChromicPDF integration tests seem fine, as ChromicPDF is started using ExUnit's start_supervised!/2 helper.
  • Our other integration tests suffer from the same issue though, as there ChromicPDF is launched as part of the host app's supervision tree, which apparently is not cleaned up properly by ExUnit.
• Both of the above issues seem to be mostly pressing for development, as on production we don't care as much about the leftover processes. Depends on the deployment environment though, if someone is running this on bare metal, they might care a lot, actually. But I'd argue that on production it's the user's responsibility to terminate their supervision tree gracefully... not sure.

ideas

• For the purpose of fixing iex commands, the user could conditionally start ChromicPDF only when iex isn't started (Code.ensure_loaded?(IEx) and apply(IEx, :started?, [])). However, this would affect iex -S mix phx.server as well which I believe some people use to pry into the running server, so this would be a bit surprising perhaps. Also it doesn't help us with mix phx.server at all.
• on_demand mode. This is currently my favourite by far: I could add some logic to the Browser module to basically turn ChromicPDF into puppeteer, spawning Chrome only when it is needed and killing it immediately afterwards. Assumption is that the 0.5s delay is acceptable for development, especially when in some (rare 😄 ) situations you don't ever need to generate a PDF, e.g. in iex shells.

Wdyt?

from chromic_pdf.