joernio / ui Goto Github PK

Right now certain tasks like parsing a long method list takes time to complete. When this is running on the ui thread, the ui freezes until this parsing completes (which can take very noticeable time).
We where able to switch the executing to a different thread using a webworker, and this solved this issue.
On a broader sense however, it is time to start thinking about how to refactor the whole application to take advantage of the possibly multi-threaded environments that it will be running on for parallel code execution.

We currently have a GUI that shows the basic result of script(rule) if the script. We need to start working on creating a much more involved reporting system for results generated by rules scripts (.sc files)

• explore the possibility of rewriting the https://github.com/joernio/joern2sarif repo to a simplified javascript script. This will be used to convert joern finding json to sarif json.
• Extend the current reporting UI that uses Rules JSON format so that it can show summary of all results in one go from multiple rules execution. Currently each script run opens a single tab for each script report. This means that if we run 20 scripts at once, we will have 20 tabs for the script results. There need to be a way to aggregate the result of all the scripts ran in one tab.
• Show Joern Findings Format results in sarif json format using SARIF Viewer (https://github.com/microsoft/sarif-web-component). since some scripts might generate findings as well as output.¹

Mockup

References

1. Rules JSON Format
This is the resulting JSON after scripts execution. If a KV pair is missing, ignore the results in UI:
{
"Title":"This is the title",
"Description":"This is the description",
"Recommendation":"This is the recommendation",
"Result": "FAIL|PASS",
“location": [
{
"methodName":"method_name",
"fileName":"Filename.c",
"lineNumber":"1",
},
{
"methodName":"method_name",
"fileName":"filename.c",
"lineNumber":"3",
},
]
}

2. Joern Findings Format

• https://raw.githubusercontent.com/joernio/joern2sarif/main/test/data/sample-findings.json

Earlier I had developed circuit as the first electron GUI for cpgql-server (GIF below). Here is the query & response interface I had done. In last few meetings @ursachec @NdibeRaymond and I had discussed about bringing that kind of more intuitive/simpler UI to replace the emulation we are trying to do in the terminal UI (which I feel is non-intuitive). The reasons we discussed were:

• Users using emulated terminal would get tricked into thinking this is real Joern/ocular terminal and would expect all kinds of features that real Joern/ocular does (tab completion, colors, left and right navigation etc.)
• We would need to support each and every feature (reformat strings coming from responses in a terminal-like way for ascii tables, lists in scala etc.) whereas we could spend time in reformatting them in more rich UI elements like chips, pills, code blocks, nice tables components etc. to improve experience. For example, tab completion could actually give a pop-up list of common queries etc. for autocomplete. Buttons could be added to click and run queries..

Hence, lets provide a basic interaction UI as default as shown below with a toggle in settings to switch to terminal like view. Code and styles can be lifted from here: https://github.com/joernio/circuit

right now the distinction between the main and the renderer process is only in code. the folder structure doesn't clearly indicate this, and this will confuse anyone who is working on the codebase for the first time.
The development setup is also clunky and you need to rebuild every time changes are made.

What should be fixed

• Refactor the application to more accurately differentiate between main and renderer process.
• Refactor the webpack build process to separate the webpack configs for the main and renderer process, and for dev and production builds.
• Add auto reloading of the main and renderer process in dev environment.
• Separate dev node_modules and node_modules needed for application packaging.
• Add proper linting.

• Any other minor change

So I've begin testing this with Ocular and the client seems to work ok there as well. I think we should have a way to get the branding etc being used driven by the server being used use Joern as a default. This means doing just a few tweaks:

• Change JOERN banner from the query terminal as default to OCULAR or JOERN based on the server
• Server needs to implement a feature which can inform client if its Ocular or Joern. Investigate if this is possible.
• Remove Joern Client from titlebar
• Decide on a name and package the client with its own name so in releases we don't have joern-desktop-client as the package name. Maybe we call it gooey and deprecate the old client repo?

A rule is a specific script which runs and generates a pass or fail result along with additional metadata. Currently in the ui we can pass arguments to script(s) at once, run the script(s) and get the result of the scripts.
To improve on this feature, we need to add support for:

• A configuration file which contains information about each rule:
  The format of the configuration file will look something like this

    {
          "title": "Hardcoded Secrets Detection",
          "Id" : "001",`
          "Filename": "/home/joern/scripts/secrets.sc" ,
          "Description": "Detect hardcoded secrets in Source",
          "Tags" : ["cwe-789", "cve-2021-2787"],
          "Language" : [ "C", "Java", "Javascript" ]
    },
    {
          "title": "Own title",
          "Id" : "001",
          "Filename": "/home/joern/scripts/script_name.sc",
          "Description": "Description goes here",
          "Tags" : ["cwe-789", "cve-2021-2787"],
          "Language" : [ "C", "Java", "Javascript" ]
    },
]

¹2. A runtime configuration which decides how scripts will be run:
Format will look like this:
[
{
"id": “001”,
“Filename”: “/home/joern/scripts/secrets.sc” ,
"Input": [
payload: “/path/to/js_code”,
payloadType: “JS”,
Resultfile: “/tmp/RESULT_secrets.json”,
],
“Output” : “/tmp/RESULT_secrets.json”,
},
]

• A GUI to manage configuration (runtime config + scripts config) ¹

• integrate most functions in the scripts view into the rules management UI and remove the scripts view.
• make it possible to add, edit and execute rules.
• make it possible to filter rules by languages and tags.
• indicate when rules are open on the open editor tabs.

Mockup

Additional Thoughts

• The GUI to manage configurations should probably have an option to export and import configurations as well.
• When a user clicks on the methods in the results, the ui should open the file containing the method and highlight the method being referred to.

To Reproduce

• start the ui and the server.
• try opening a project or switching to a different project.

Expected Behaviour

The new project should open, and the active folder should change to point to the newly opened project's folder.

What happens

The new project opens, but the active folder section remain unchanged.

This is a sketch, will update as I do figure out more about where to make improvements:

• Consider using "window" technique.

As the DOM get's larger, the ui tends to get slower. This indirectly translates to "The more time spent using the ui without closing, the slower it gets". This shouldn't be so. ways to fix this includes:

• research on how "window" technique can be used to only have nodes that are enough to cover the viewport in the dom at a given point in time.
• Research already existing solutions that can solve this issue? (react-window,
  react- virtualized, any other solution?)
• What happens if the existing solutions can't handle the complexity of our dom? build in-house solution?

• Investigate how Json parsing is currently being done and find better solutions.

• use lodash cloneDeep vs JSON.parse(JSON.stringify()) vs custom deep clone method? actual bench-marking will be useful here. JSON.parse(JSON.stringify()) seems to be the slowest from experience.

• Ensure that state mutation is not taking place anywhere in the app.
  Before assigning value to object read from store or state, always deep clone the object.
• Spend some time in the gut of the application and try to weed out inefficiencies in the way server interaction is being done.
• Try to ensure that useEffect only listens to values that are absolutely necessary for it's operation and nothing else.

We need to provide a finding json sample to @NdibeRaymond so he can begin experiments with joern2sarif. There are sample Finding JSONs, but we need to give more realistic and simple JSONs coming from rule scripts

we should show a small debug screen somewhere (for example in the bottom-left corner near the setting screen) containing all the queries which went out to the server, and the elapsed time since the start of their execution; at the moment, the "Running" icon is shown after having triggered multiple commands in a row, and I can't tell which one is still executing