rabbitmq / horus Goto Github PK

It's possible to use erpc:call/2 or erpc:multicall/2 to execute an anonymous function remotely. However, it relies on the fact that the module is available on the remote node.

It would be nice to cover the use case where the remote node does not have the module where the anonymous function is defined. Horus could extract the function and load it on the remote node.

The API could look like:

-spec exec(Node | Nodes, StandaloneFun, Args) -> Ret when
      Node :: node(),
      Nodes :: [node()],
      StandaloneFun :: horus_fun(),
      Args :: [any()]).

Or remote_exec/3 or rpc/3 if exec/3 is too confusing vs. exec/2.

Is your feature request related to a problem? Please describe.

Horus currently uses the beam_disasm module from the compiler application in Erlang/OTP to disassemble (convert BEAM chunks into instructions). beam_disasm had some bugs involving mistranslated instructions (for example erlang/otp#6447) which are papered over by Horus (for example see pass1_process_instructions/3). These mistranslations are fixed upstream but we must wait until Erlang/OTP 26 to take full advantage of the upstream fixes and drop compatibility code in Horus.

Also, beam_disasm in Erlang/OTP 26 cannot decode modules compiled with the Erlang/OTP 25 compiler. Erlang/OTP 26 introduced a new format for the Type BEAM chunk and also dropped all code for serializing and deserializing the version 1 type chunk from Erlang/OTP 25. This could happen again to this particular chunk or to any other BEAM chunk. A custom disassembler could support multiple BEAM chunk versions.

Describe the solution you'd like

Horus could implement a custom disassembler based on beam_disasm. This would allow us to fix the mistranslations in-tree and drop the compatibility code now. We could also support multiple BEAM chunk versions and have fine control over which Erlang/OTP versions are supported.

The downside to a custom disassembler is that it's more work to maintain. beam_disasm in Erlang/OTP provides a good guide but sometimes has bugs as mentioned. This part of Horus would need some work for every Erlang/OTP major release that introduces new instructions or changes the format of BEAM chunks. (Every major release since Erlang/OTP 20 has added new instructions.)

Describe alternatives you've considered

We could restrict the versions recommended for running Rabbit releases that include Horus and Khepri (see https://www.rabbitmq.com/which-erlang.html) to work around the version compatibility problem. The compatibility code for mistranslations could stay in-tree until Horus depends on Erlang/OTP 26 as a minimum.

Additional context

No response

Why

Currently, when an anonymous function is extracted from a cover-compiled module, we loose the cover-compilation because Horus takes the assembly from the original module, not the cover-compiled copy.

How

It should be possible to use the cover-compiled module assembly instead to keep cover support.

Why

Horus relies on the availability of erts to initialize the list of modules to skip when it needs to decide if a call should be followed to extract a function or if the call should be preserved as is.

It already depends on kernel and stdlib for instance. I assumed that erts was implicit as I don't see how Erlang can work without it. It looks like if erts is not included explicitly in an Erlang release file or in the applications it bundles, it is excluded from the release (the erts Erlang application, not the actual runtime). This breaks Horus because it won't know about e.g. the erlang module and will try and fail to extract functions from that module.

See rabbitmq/khepri#212 and rabbitmq/khepri#222 for users who encountered this problem. A function extraction will fail with {horus_ex, module_not_found, #{module => erlang}} if erts is missing from a release.

How

It looks like a bug in the way releases are created because, to me, it makes no sense to not always include erts (like kernel and stdlib).

As a workaround, we should include erts in the dependencies of Horus.

Why

I guess the maybe expressions come with new Beam instructions and we don't handle them right now.

How

We at least need to add a testcase to make sure horus can extract them, and add support for the related instructions if there are.

Before OTP 27, cover worked by inserting code to do counters:add(CRef, LineIndex, 1) around any significant code. The CRef is a counter for the original module's coverage, so Horus only needed to preserve those instructions in the standalone module in order to have the function's coverage count for the original module.

OTP 27 introduces "native" coverage (introduced in erlang/otp#7856). The compiler now emits executable_line instructions when passed the force_line_counters (or line_coverage option, depending on the system's code:get_coverage_mode/0). The JIT turns that that instruction into an atomic add or literal mov that updates a section of the BeamCodeHeader's coverage buffer (see x86, arm). BeamCodeHeader is a member of each module instance (erl_module_instance) and the BIF that cover uses to read coverage information when native coverage is enabled reads from this new buffer from the current module. (As a consequence, old coverage is now no longer accessible after upgrading/reloading a module.)

The old cover code is still around for architectures that use the emulator but we can't turn off native coverage on a JIT-enabled build of OTP.

Since coverage information is now attached to module instances, I don't believe we can trick cover into counting executions of horus functions anymore.

Why

Currently, if an extracted standalone function throws an exception, the stacktrace will point to the generated module. Therefore, the stacktrace will have very little sense and help the developer to locate the real line of code which failed.

How

It would be nice if the generated module could rewrite the exception's stacktrace to point to the original source code.