parcel-bundler / parcel-benchmark-action Goto Github PK

Figure out why Parcel crashes on react-hn

Benchmarking against other bundling tools (rollup, browserify, webpack, ...) would be cool.

Not sure if it should be done of every PR though...

We could however make a script that uses the same utils that could just run from this repo if anyone wanted.

Splitting this action into an api and action would allow us to deploy this on a properly managed machine. Making our performance tests more reliable.

The bundler plugin can change in a PR, to prevent this action from crashing or giving invalid data. Ensure the bundle graph is identical.

In case it isn't, we should just log bundlesizes without any diffs.

Testing the application in watch mode or serve mode will be slightly more complicated than just build.
But would allow testing performance of dev specific elements like the watcher.

Currently we don't have a lot of test applications.

We mainly need a very big and complex application to test on. Small ones are also welcome :)

Using a seperate API to interact with github and store benchmark results would allow us to create a web-interface for the benchmarks

It would also allow us to plot performance changes over time.

Another advantage would be that our actual interaction with the github API would be split from github, aka logs about the api aren't public, github action logs are (resulting in a probably more secure environment)

Apparently Github Actions run from the source branch not the target branch, so the apikey is undefined if a PR compares with a branch outside of the Parcel org.

Not having an API key is not an option unless we run the benchmarks on a seperate server. Related which might happen, see: #8 but than we'd also need to verify the source to ensure no-one can submit incorrect benchmarks

Currently all benchmarks are put into the parcel monorepo that gets cloned, so it can use yarn workspaces for linking everything up nicely.

Figure out a way to link up everything manually so we have a more reliable environment. (prevent pkg version conflicts).

Github actions isn't really reliable for performance tests. We should probably run these on a dedicated vm/machine

Test this out on a Parcel fork and figure out how to post a comment with the benchmark results

https://github.com/parcel-bundler/parcel/pull/5414/checks?check_run_id=1546225252

🚨 Build failed.
Error: Entry /usr/src/app/.tmp/three-1607880853233/src/Three.js does not exist
Error: Entry /usr/src/app/.tmp/three-1607880853233/src/Three.js does not exist
    at EntryResolver.resolveEntry (/usr/src/app/.tmp/main/packages/core/core/lib/requests/EntryRequest.js:101:13)
command node_modules/.bin/parcel build ./src/Three.js --log-level warn --no-cache failed with status code 1.
Cleanup benchmark environment
command node_modules/.bin/parcel build ./src/Three.js --log-level warn --no-cache failed with status code 1.
Unlinking package @parcel/reporter-build-metrics
success Unregistered "@parcel/reporter-build-metrics".

https://github.com/parcel-bundler/parcel/pull/5462/checks?check_run_id=1542100126