privatenumber / minification-benchmarks Goto Github PK

Helps with understanding which minifiers are maintained

Hey there,
this package is a Rust library which comes with JS bindings - is this something you would consider adding to your benchmark?

Cheers
S1SYPHOS

uglify-js supports ES6+ syntax now up to ES2020 and appears to minify the two libraries that previously failed:

$ node_modules/uglify-js/bin/uglifyjs -V
uglify-js 3.13.1

$ node_modules/uglify-js/bin/uglifyjs node_modules/d3/dist/d3.js -mc | wc -c
  267653

$ node_modules/uglify-js/bin/uglifyjs node_modules/terser/dist/bundle.min.js -mc | wc -c
  407144

Might you be able to update this benchmark to the latest release of uglify-js?

--- a/package.json
+++ b/package.json
@@ -41 +41 @@
-               "uglify-js": "^3.12.5",
+               "uglify-js": "^3.13.1",

Unfortunately I'm not running pnpm so I can't give you a PR.

Thanks in advance.

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

This repository currently has no open or pending branches.

Detected dependencies

• Check this box to trigger a request for Renovate to run again on this repository

I believe the only minifier that didn't support stripping comments was esbuild, but with v0.11.15 just released, now all minifiers can strip comments.

Thanks for publishing these benchmarks. I'm the author of bundlephobia (a tool developers use to find out minified and minzipped sizes of npm packages) and was contemplating on replacing Terser with ESBuild's minifier to shave a few seconds from build.

Seeing this benchmark, I got the picture that ESBuild and Terser and neck to neck. However, this doesn't seem to be the case for large packages. While the minified size might still be comparable in some cases, the gzipped size difference between esbuild and terser can vary between 5%-9% — which one might consider non-trivial difference for minification.

Benchmarks after replacing Terser with ESBuild in Bundlephobia's webpack config, and after manually extracting out license-style comments —

The results remain the same whether running esbuild minify through esbuild-loader or the ESBuild CLI on the webpack built bundle.

Is there anyway to capture this on the current page?

Feature request

There's a very big difference in minifying output for a ES2018 target or a ES5 target.

This is ES2018:

var foo = 1;
var bar = 2;
{ foo, bar }

And this is ES5:

var foo = 1;
var bar = 2;
{ foo: foo, bar: bar }

Currently, not all tools are minifying to the same target. For example, Terser has the default target of ES5 while esbuild has the default target of ES2018.

This means that the current results are not accurate and do not represent what each tool can accomplish.

Alternatives

No response

Additional context

No response

Contributions

• I plan to open a pull request for this issue
• I plan to make a financial contribution to this project

from the readme:

Benchmarks are gathered on every PR via GitHub Actions (verifiable minified artifacts are uploaded on each run)

i'd expect that because github actions has varying use throughout the day that the performance varies massively and could spike and dip at any time. it's possible this is a useful metric for other people that need to run minification fast on github actions but generally performance is more often looked into by people minifying on their own machines, whether that's during testing, a local build step before uploading to a server, or automating a build step on a personal weak server.

maybe this repository should try benchmarking on user hardware? it probably wouldn't get updates as fast but would likely be much more useful to have benchmarks done on things like raspberry pis, desktops, old laptops, etc.

the program is extremely buggy, it couldn't install to my working directory because there were spaces in the name, and npx bina --help throws an error message despite npx bina saying to use bina --help
it also has terrible security practices, like relying on the bina.egoist.sh domain to run a shell script without any verification whatsoever, when really it should just be part of the npm package

i'm assuming bina is only used for the postinstall to install tdewolff/minify, this seems unneccecary considering minify has an npm package.
it mentions to use npm i @tdewolff/minify for javascript bindings on the github readme

Feature request

Bench:

• https://github.com/mrclay/minify/
• https://github.com/douglascrockford/JSMin
• https://github.com/mrclay/jsmin-php

Motivations

Missing popular js minifiers

Alternatives

No response

Additional context

No response

Contributions

• I plan to open a pull request for this issue
• I plan to make a financial contribution to this project

4e48512

I checked out the parent commit and updated @swc/core to v1.2.189 and it works

It would be interesting to see whether bun build is capable of bundling any of the libraries and how well it does.

For all links uglify-js who lead to : https://github.com/privatenumber/minification-benchmarks/blob/master/lib/minifiers/uglify-js.js
Well, all minifiers links in fact.

similar to .nvmrc file
currently .nvmrc contains v16.15.0, .node-version should have 16.15.0 (no v)

Quick question. Is the "Time" column on each table how long it takes the program to minify the code or how long the browser takes to parse/execute the minified code? 🤔

https://www.npmjs.com/package/@tdewolff/minify it's like a toy project, with 62 Weekly downloads.

It looks like you don't use compilation_level - ADVANCED_OPTIMIZATIONS for Google closure compiler case
https://developers.google.com/closure/compiler/docs/api-tutorial3
I think because of this this compiler is not best of all in terms of code compression )

The option exists, but it's there because of future plan.
The only thing swc does and which is similar to minification is skipping comments and some whitespaces.

I'm porting terser to rust at swc-project/swc#1302
I think swc should be added to the list after it's implemented.

And swc.transform applies (really) lots of compatibility transforms, as swc reimplements features of babel.
I'm going to add swc.minify with the PR I mentioned, so it should be used instead of swc.transform.

Feature request

Every upgrade PR doesn't need to re-benchmark everything—just the new minifier. Each minification result should be cached by minifier name, version, and output code.

Motivations

Slow GitHub Actions taking ~30min.

Alternatives

No response

Additional context

No response

Contributions

• I plan to open a pull request for this issue
• I plan to make a financial contribution to this project

The "hyperfine" tool at https://github.com/sharkdp/hyperfine could be useful for abstracting benchmarking logic. To use it with minifiers lacking CLIs, we'll need to create simple CLI scripts (not difficult). In such cases, the benchmarks will include FS read, but it might have a negligible impact and lead to a fairer comparison, as all minifiers will have to read from the FS, reflecting the performance cost based on runtime choice.

https://github.com/swc-project/swc

Hi @privatenumber , what do you use to verify the validity of the minification output? It says "TypeError" for all minify/tdewolff outputs, but loading the minified output in Firefox or Chrome works fine (no errors), and putting it in ESLint gives no relevant errors (tested with moment.js).

Is the library outdated? Have you tried using https://www.npmjs.com/package/@tdewolff/minify?

The ADVANCED mode of the Closure Compiler is pretty strict, and any error make it fails. You'll need to provide sufficient information about all variables used in the library to minify in order to use it.

As you can't of course do that here, one (very) ugly way to avoid failure of minification would be to set the jscomp_off flag to '*' in google-closure-compiler.advanced.js.

I tried running it and here are the results, however I didn't test each library to see if it works. What do you think about it?

📋 Results

d3 v6.7.0

• Unminified size: 560.0 kB
• Unminified Gzip size: 131.4 kB

jquery v3.6.0

• Unminified size: 288.6 kB
• Unminified Gzip size: 85.1 kB

lodash v4.17.21

• Unminified size: 544.1 kB
• Unminified Gzip size: 97.3 kB

moment v2.29.1

• Unminified size: 173.9 kB
• Unminified Gzip size: 36.5 kB

react v17.0.2

• Unminified size: 72.1 kB
• Unminified Gzip size: 19.5 kB

terser v5.7.0

• Unminified size: 869.4 kB
• Unminified Gzip size: 174.5 kB

three v0.124.0

• Unminified size: 1.2 MB
• Unminified Gzip size: 249.0 kB

vue v2.6.12

• Unminified size: 223.2 kB
• Unminified Gzip size: 62.3 kB

by comparing compressed ratio with time

Request from @kzc:

• Each "Time" is annotated with a multiplier relative to the fastest minifier

@privatenumber It'd be useful if both the Minified size and Minzipped size columns followed a similar methodology and showed a multiplier (or percentage) relative to the smallest value in the column. The current percentages relative to the original file and gzipped file sizes are not as useful.

For example, the minzipped size for d3 shows -33% 87.23 kB for uglify-js and -31% 90.63 kB for esbuild - a two percent spread in size relative to the original gzip size. But most users are interested in the relative size difference between the bundles to show how much they could be saving over the wire - in this case: 90.63 / 87.23 = 1.039x - nearly a 4% difference between the two minifiers: (90.63/87.23 - 1) * 100 = 3.90%.

846a888#commitcomment-55713496

https://github.com/tedious/JShrink

this minifier is used in things like cobalt through html-minifier, it would be helpful to have benchmarks of these so i can know if i should use the builtin minifier for js or use it alongside swc

We used to have an analysis section but I removed it because it was unmaintained.

I'm thinking the benchmarks could also output a JSON/YAML version, and we could pass that into ChatGPT to generate an analysis that recommends a minifier for different use-cases.

It should probably live in a different GitHub Action, because generating the benchmarks is quite expensive (40 min), and we wouldn't want the action to fail due to an API error (which happens often).