dependents / node-dependents-editor-backend Goto Github PK

Currently does nothing.

• Shell out to bin using a flag to indicate the command to execute
• Consume the shell output

Use https://github.com/mrjoelkemp/Dependents as a reference.

Then test it.

Need to make sure the bin result printing works as expected

Passing handlers all the way down means you need to traverse the dep tree upwards to find the definition site.

jump to definition (see of sublime even knows what's going on)

Port of dependents/Dependents#132

Context menu option for the current file.

We can detect the type of the current module and create a new view with a default test file.

Assumes that an amd module is using a require mechanism for testing.
Depends on dependents/Dependents#129 to generate the module name to pull in.

Ref dependents/Dependents#212

This would let a user know whether or not there is a mistake in their .deprc file.

• This should only print the error message, not stack trace.

Babylon spits out a horrific stack trace. We should intercept that and show a message indicating that there's a syntax error on the page.

Given a driver, identify the important scripts.

Importance could be defined in several ways, but I'm thinking about in terms of degree (number of connections in terms of modules that require it).

We'd like to say "these modules are the most used" throughout an app/feature.

Ref dependents/Dependents#97

Ref dependents/node-sass-lookup#1

Like a filter applied to the normal dependency tree

The filter would parse out the file into an AST and sniff it to determine if it's a React component

Port of dependents/Dependents#172

Menu option that uses node-app-root to detect top level dirs

Use https://github.com/xavi-/node-copy-paste for copying results to clipboard

Put it under the --copy-find-dependents cli flag

Editors should pass an entire line down (needs to be escaped for cli usage) and the backend trims it into an import for use in a cabinet lookup.

This will allow plugins that need config data to retrieve it via shelling out and json parsing the output.

Example usage:

• Removing the root/directory of the results to be shown in a file select dropdown. Still need the full paths for filesystem resolution.

Ref dependents/node-dependency-tree#29

All other features should get this support

Deferred until that lib is better

Port of dependents/Dependents#191

Ref dependents/node-taxicab#15

https://github.com/team-gryff/react-monocle

• Very interactive
• Includes on-hover details of the modules

• Find all listeners of an event
• Find all triggers of an event
• Open all files that care about an event

Ref dependents/Dependents#52

Jump to definition

• Get absolute position of clicked char
• Parse file
• loop through all nodes
• see if there’s a node that collides with the clicked position
• if that node is a variable, sniff if it’s an import
• if it’s an import jump to that dependency
• If it’s a method of a dependency, jump to that method

ES6:

Support jumping to Uploader in import { Uploader } from 'components/common’;
Support jumping to CoverArtContainer definition in

<CoverArtContainer
                image={image}
                clickthroughUrl={clickthroughUrl}
                onChange={onChange} />

This belongs at the resolver level. For es6/commonjs, we currently don't have to check if the resolved file exists.

Refs dependents/node-module-lookup-amd#8

The root is noisy

This could model the default copy path behavior. Might be annoying to try using the feature on a file not covered by the js or styles root and see an error.

Ref dependents/Dependents#125

This is perhaps a more useful subset of the "View dependency tree" feature

Two features here:

• View subclasses (Who extends me)
• View ancestors (Who do I extend)

Both features yield a JSON output.

If we want to make this opt-out, support an opt-out flag in the deprc that prevents tracking.

Update readme to make it clear that anonymous data will be collected on performance and command usage to improve the tool and track errors.

Try to reuse mixpanel. Similar to what the sublime plugin currently does.

• Account for no internet connection

Ref dependents/node-dependency-tree#1

I have access to the immediate dependents and the relevant entry points, but I don't have a way to see all the steps in between to have better context.

Users would currently need to find the relevant drivers then view the tree for those drivers. That gives irrelevant paths though.

foo.styl does @import bar/*;

Expected: If we have bar/baz.styl, foo.styl should be a dependent of bar/baz.styl.

Refs dependents/node-stylus-lookup#1

Ref dependents/node-detective-es6#8

We want to have it use the newer version of filing-cabinet and precinct to get alignment across versions to slim down the client installs.

Motivation

Currently, the backend is modeled after a composition of stateless functions. Every Dependents operation is meant to be standalone and can handle all work needed for fulfilling its purpose. This is great for simplicity, but results in poor performance. Every subsequent operation has to regenerate its local state – which could mean re-processing (and generating ASTs for every file) an entire directory.

AST generation is the bottleneck. On average, every file in the processed directory (either root and/or styles_root) takes a few hundred ms to parse. Even with parallelization in place, the hundreds of ms stack up to cause a few seconds of wait time. In large codebases (like ESLint), this means the user waits up to 19s for the results of a single command (ex: Find Dependents).

Example

Assume that we have 200 files in a directory. Assume each file takes 100ms to parse (which would take some heroic and hopeful parser optimizations, but also depends on a user's machine specs). In a serial processing fashion, a command would take 100ms per file * 100 files = 10,000ms (10s) to generate the ASTs for every file (and then have to do the sniffs, like finding the dependents of a given file, based on the operation being requested).

If we split the work across the 4 cores of a typical user's laptop, that means each core is responsible for 50 files. Since each core processes its batch of files serially, each core will theoretically take 100ms per file * 50 files = 5000ms (5s). Since all cores process their batches simultaneously, it takes roughly 5s to generate the ASTs for all files. This cuts the runtime in half, but this is still an O(n) operation; when we parallelize the work, we're only multiplying n by a fraction. As we add more files to the project/directory, the wait time of 5s increases.

We do the above work every time a user issues a command. Note, some commands like Copy Path and Jump To Dependency don't require AST generation.

A more efficient architecture involves doing the AST generation work once and reusing the ASTs on subsequent commands. Of course, this requires monitoring the filesystem for file changes (saving a file, adding new files, removing files, adding/removing directories) that may require regenerating a file's (or a directory of files') AST.

If the runtime of a command like Find Dependents is O(nk) (where n is the cost of generating the ast for every file and k is the cost of sniffing the generated asts to find the dependents of a file), then we will pay that cost on the first instance of the command, but only pay O(k) on subsequent requests.

The gist

Client (on startup) instructs the backend to init

• Create an http server that listens for GET requests containing data describing the operation request
  • Maybe support a configurable port (nice to have)
• Construct a (long-lived) dependency tree instance for the current directory
  • Do root and styles_root even matter anymore?
  • We could join those paths together in a glob for all files containing support extensions
• Once the server is ready and the dependency tree has been generated, we can accept operation requests

Client issues commands to the server

• The server expects http requests
  • Every client would need to use their host language's http library to issue requests
• It would be great to not have to change clients to issue http requests and have the backend cli translate cli commands into http requests
  • Can't have the cli talk to the server via an object instance since they're on different processes.

Things to consider:

• Server idle shutdown
  • Debounced on every command request
  • Does the cli need to attempt to ping the server and start a new one if it gets no pong?
• Filesystem updates?
  • Adding files
  • Removing files
  • Removing directories (empty or with files)
  • Adding directories (empty or with files)
• What if the user modifies the deprc while the server is running?
  • Should we reparse the config on every request
  • This would imply supporting every config value change (adding/removing directories) and modifying the dependency tree object appropriately.
  • If we have a dependency tree object for the entire project (minus the vendor excludes), there may not be a need to handle config changes

References

• Ternjs server startup via cli script: https://github.com/ternjs/tern/blob/95dfbf405802207b5529314b83b6104f9ea9b042/bin/tern
• Atom-tern client invoking commands on the server: https://github.com/tststs/atom-ternjs/blob/6344a6b63d2ba33d7b56266952e7ed073dce45eb/lib/atom-ternjs-server-worker.js#L34
  • Because it's a JS env, that client opts for issuing commands to the library-layer server instead of the server process.

Note: this is for files with the .sass extension, not .scss.

Trying to fetch the dependents of styles2.sass yields no results.

• Doesn't work for files in the same directory
• Does work for finding the dependents of a subdirectory module

Failing test:

    it.skip('finds dependents within the same directory', function() {
      return this._run({
        filename: `${this._directory}/styles2.sass`
      }).then(results => {
        this._assertSomeDependent(`${this._directory}/styles.sass`, results);
      });
    });

When a user wants to view the tree, they should see an actual visualization, not a JSON dump.

Generate a webpage that renders the tree with the data generated from node-dependency-tree.

Then there is no need for #33