Redesign executable layout: separate electron and backend processes about zap HOT 9 CLOSED

👍 The plan to split out the UI features from the pure-command-line zap core sounds good! 👍

from zap.

(NOTE: This comment has been mostly superseeded by further comment...)

Ok, we have a better design now:
1.) Split up the server executable and the UI executable.

• Server executable becomes a full daemon running all the server aspects.
• UI executable is then an electron container, essentially just serving as a simple browser to the server.

from zap.

(NOTE: This comment has been mostly superseeded by further comment...)

I'll use this bug as a placeholder for the redesign that will achieve the following endgoal:

Zap will be shipped as TWO separate executables.

1.) zap will be a pure node.js executable. There will be no electron there. The executable will perform everything zap does now, with the same command lines, except that all the UI options will be taken out. It will run as "server" by default.

2.) zapui will be an electron executable. It WILL NOT CONTAIN any of the logic zap has. It WILL NOT open a database. IT WILL NOT query the database. It will only do the following:
a.) When needed it will execute zap server.
b.) It will open the new window, or a separate window to edit the files provided.
c.) It will run in "electron single instance mode" which will make sure only one copy of zapui runs, and the main process manages with dispatching of the renderer processes.
d.) Renderer processes communicate with zap via HTTP.
e.) Main process communicates with zap via RPC.

from zap.

Here is the plan as I see it right now:

1.) Current state:
One zap executable, which sometimes acts as a CLI app, sometimes as a UI app and sometimes as a http server or something as all the things in one... There is also few scripts that can serve as facade to this adding confusion to it, while helping sometimes.

2.) Desired state after redesign:
a.) zapd daemon, which is a pure node.js executable that essentially runs HTTP server, DB connection and any IPC we do.
b.) zap executable, that is for use in CI, and is a command-line only small executable that passes data to zapd and headlessly does things that don't require UI. (And is able to fire up zapd if it's not yet running.) It's a pure node.js executable, no electron.
c.) zapui executable that is essentially just an electron browser packaging. Doesn't contain any of the logic that is in zapd . Has no database layers. Just parses few command line arguments, and fires up electron packaged browser, possibly starting zapd if it's not yet running, and then displays the UI.

We will have to rework few layers, but mostly we will have to change the packagings.

This is to fix several problems we are facing:

• occasional core dumps from zap, which always come from electron, in an environment where electron is not even needed.
• the problem of DISPLAY being needed. In this workd ONLY zapui needs DISPLAY, but zapui is not needed at all in Jenkins environment.
• cleanup of our "server mode", which is at this point a bit ugly mostly because the one-and-only zap executable is trying to do EVERYTHING.

Alternative ideas, comments and complaints welcome. Chocolate even more so....

from zap.

@bzbarsky-apple , @tcarmelveilleux , @andy31415 or anyone else who cares about zap architecture, please read at least this last comment, and comment if you see issues with this.
Most of this came from the lessons learned with zap in ZigbeePro environment, and this approach would make things way faster, stabler and leaner.

Any additional suggestions welcome.

In the wake of this, there will certainly be some changes to the way Matter executes zap (those python scripts), but they should not be major.

This also prepares zap much more to be included as a plugin to VSCode or packaged along with however Matter SDK will ultimately be packaged.

from zap.

The above plan would definitely help move code gen to CI and reduce code gen merge risk

from zap.

I believe from a step forward perspective it would be progress.

However from a full solution from a code generation perspective, my ideal tool is something that takes input data and generates output. In this case 'cluster data + templates => cpp source output'.

Mixing daemons seem to complicate the environment. Other tools do it (e.g. bazel) however I believe that is for caching reasons and I would generally frown upon daemons. Having a build system that starts daemons and http servers sounds very complex for something that seems to want 'data + template = output'.

from zap.

Comment from folks who want to use zap on the Raspi:

• we should separatelly package zapui and zap + zapd for Linux.
  So Linux installers should be twofold:
• zap.rpm (or zap.deb), which packages zapd and zap and has NO UI dependencies.
• zapui.rpm (or zapui.deb), which packages zapui and DOES HAVE UI dependencies through electron.

from zap.

Assigning to @jingteng25742...

from zap.