Giter Club home page Giter Club logo

zephyr's Introduction

zephyr

Maintainer: MaybeJustJames zephyr

A tree-shaking tool for PureScript. zephyr takes root terms, finds all terms which are required to evaluate them, and generates code just for them. This is done across all dependencies, and can substantially reduce the size of PureScript bundles. zephyr can also evaluate some expressions (an experimental feature).

Installation

The simplest option is to download the latest release binary. You may also build from source (see "Build & Test" section below).

Usage

# compile your project
purs compile -g corefn bower_components/purescript-*/src/**/*.purs src/**/*.purs

# run `zephyr`
zephyr -f Main.main

then you can bundle with purs bundle command:

purs bundle -o app.js -m Main dce-output/**/*.js

You can integrate it with other build tools, see below.

You can specify modules as entry points, which is the same as specifying all exported identifiers.

# include all identifiers from Data.Eq module
zephyr Data.Eq

# as above
zephyr module:Data.Eq

# include Data.Eq.Eq identifier of Data.Eq module
zephyr ident:Data.Eq.Eq

# include 'Data.Eq.eq' identifier
zephyr Data.Eq.eq

zephyr reads corefn json representation from the output directory, removes non transitive dependencies of entry points and generates common js modules (or corefn representation) to dce-output directory.

Evaluation of literal expressions

Zephyr can evaluate some literal expressions.

import Config (isProduction)

a = if isProduction
  then "api/prod/"
  else "api/dev/"

will be transformed to

a = "api/prod/"

whenever isProduction is true. This allows to have different development and production environments while still ship a minified code which only contains production code. You may define isProduction in a module under a src-prod directory and include it when compiling production code with pulp build -I src-prod and to have another copy for your development environment under src-dev where isProduction is set to false.

Integration with build tools

zephyr can be integrated with

  • pulp: use pulp build -- -g corefn to compile, and pulp browserify --skip-compile -o dce-output to bundle zephyr's output.
  • parcel
  • spago. See this example. Use spago build --purs-args '--codegen corefn,js' to compile, using spago bundle is currently affected by issue.

Build & Test

cabal build exe:zephyr

To run tests

cabal run zephyr-test

C Libraries

The released binaries are dynamically linked against glibc, so if your system is using musl (like Alpine Linux in docker) or another alternative C library, you will need to compile zephyr from source using ghc on that system.

Comments

The -f switch is not 100% safe. Upon running, zephyr will remove exports from foreign modules that seems to be not used: are not used in PureScript code and seem not to be used in the foreign module. If you simply assign to exports using JavaScript dot notation then you will be fine, but if you use square notation exports[var] in a dynamic way (i.e. var is a true variable rather than a string literal), then zephyr might remove code that shouldn't be removed.

zephyr's People

Contributors

coot avatar dgendill avatar kl0tl avatar maybejustjames avatar milesfrain avatar rnons avatar toastal avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

zephyr's Issues

Halogen Hooks problems

after running zephyr on this recipe after clicking on plus or minus it will throws an error.

Screen Shot 1400-10-15 at 18 23 37

the problem is inside the halogen free monad hooks, but I could not find the culprit.

language-javascript version in stack.yaml's extra-deps too low?

Hello!

When attempting to build both zephyr-0.3.0 yesterday and zephyr-0.3.1 today the following errors have come up both times:

zephyr> build (lib + exe)
Preprocessing library for zephyr-0.3.1..
Building library for zephyr-0.3.1..
[1 of 7] Compiling Language.PureScript.DCE.Constants
[2 of 7] Compiling Language.PureScript.DCE.Errors
[3 of 7] Compiling Language.PureScript.DCE.Foreign

/mnt/c/code/hs/zephyr/src/Language/PureScript/DCE/Foreign.hs:135:21: error:
    Not in scope: data constructor ‘JSAsyncFunction’
    |
135 |     isUsedInStmt n (JSAsyncFunction _ _ _ _ es _ (JSBlock _ ss _) _) =
    |                     ^^^^^^^^^^^^^^^

/mnt/c/code/hs/zephyr/src/Language/PureScript/DCE/Foreign.hs:165:21: error:
    Not in scope: data constructor ‘JSAwaitExpression’
    Perhaps you meant one of these:
      ‘JSAssignExpression’ (imported from Language.JavaScript.Parser.AST),
      ‘JSCallExpression’ (imported from Language.JavaScript.Parser.AST),
      ‘JSClassExpression’ (imported from Language.JavaScript.Parser.AST)
    |
165 |     isUsedInExpr n (JSAwaitExpression _ e) = isUsedInExpr n e
    |                     ^^^^^^^^^^^^^^^^^


--  While building package zephyr-0.3.1 using:
      /home/me/.stack/setup-exe-cache/x86_64-linux/Cabal-simple_mPHDZzAJ_2.4.0.1_ghc-8.6.5 --builddir=.w/home/wsl1/dist/x86_64-linux/Cabal-2.4.0.1 build lib:zephyr exe:zephyr --ghc-options " -fdiagnostics-color=always"
    Process exited with code: ExitFailure 1

On investigation, the two mentioned constructors JSAsyncFunction and JSAwaitExpression only appear to have been included in Language.JavaScript.Parser.AST.JSExpression in language-javascript-0.7.1.0 (and were not present in 0.7.0.0).

The stack.yaml specifically refers to language-javascript-0.7.0.0 and the zephyr.cabal file has a version bound of language-javascript ^>=0.7.

For reference, I'm building with stack build --copy-bins using stack-2.3.1 on WSL1 Ubuntu 18.04.

Bumping the version in the stack.yaml to 0.7.1.0 allows everything to build fine. :)

Build fails

Simply running

purs compile --codegen corefn (spago sources)
zephyr Main

gives me this error:

zephyr: output/Affjax/foreign.js: copyFile:atomicCopyFileContents:withReplacementFile:copyFileToHandle:openBinaryFile: does not exist (No such file or directory)

Fails to parse corefn.json with 0.15.3

$ zephyr -f Main.main

Error: Failed parsing:
  output/Data.TraversableWithIndex/corefn.json
        output/Data.Semigroup.Last/corefn.json
        output/Control.Monad.Rec.Class/corefn.json
        output/Data.Monoid.Generic/corefn.json
        output/Data.Show.Generic/corefn.json
        ... (139 more)

"Tree-shaking" as a hyphenated compound adjective

The repo mentions "tree shaking tool" a lot, but grammatically it should be compound adjective as "tree-shaking tool" in the README and other places through the repo.

From Grammar Monster:

You can use a hyphen (or hyphens) to link the words in a compound adjective to show it is a single adjective. (A compound adjective is a single adjective that is made up of more than one word.) For example:

  • two-seater aircraft
  • never-to-be-forgotten experience

An existing example is in Rollup's documentation

Error in evaluator breaking code

I've been testing the spork todo-mvc example, and it is failing when using zephyrs output. The app loads, but I think there is a bad transform, particularly around pattern matches.

For example, some original output:

  var update = function (model) {
      return function (v) {
          if (v instanceof None) {
              return Spork_Transition.purely(model);
          };
          if (v instanceof UpdatePending) {
              return Spork_Transition.purely({
                  pending: v.value0,
                  fresh: model.fresh,
                  todos: model.todos,
                  visibility: model.visibility
              });
          };
  ...

And the corresponding output from zephyr:

  var update = function (model) {
      return function (v) {
          if (v instanceof None) {
              return Spork_Transition.purely(model);
          };
          if (v instanceof UpdatePending) {
              return Spork_Transition.purely({
                  pending: v,
                  fresh: model.fresh,
                  todos: model.todos,
                  visibility: model.visibility
              });
          };
  ...

In particular this diff:
pending: v.value0,
pending: v,

This is giving me a dreaded [object Object] 🙀

`-f` removes exports used by other exports

This might actually be intentional and we will need to update the FFI in the "offending" module. I'm using react-basic-hooks.
And that has code like this:

exports.useEffect_ = function (eq, deps, effect) {
  var memoizedKey = exports.useMemo_(eq, deps);
  React.useEffect(effect, [memoizedKey]);
};

I don't use useMemo_ (in fact I can't since it's not exported) but I use useEffect which in turn uses useEffect_ which then can't find useMemo_.

Would you say this is a bug or should the FFI be rewritten in a way that avoids this?

Edit: Yeah I just had a look at dce-output and that's what's happening. I will try to rewrite the FFI.

String array and object evaluation

#10, other than turning off evaluation we could

  • keep two ASTs - evaluated and the original
  • use the original one for printing js modules
  • use the evaluated to perform dce

Upgrade to PureScript v0.14

Currently this is still pointed at 0.13.8, but 0.14.x has been out for a while. It appears to work, but a banner comment appears at the top of the file saying it's generated with PureScript 0.13.8 which raises some red flags to me.

Build with Nix

As mentioned in #64, Nix makes setting up a build/dev environment much easier.

Missing imports

zephyr: An internal error occurred during compilation: "Missing value in mnLookup: App.Domain.
Route"
Please report this at https://github.com/purescript/purescript/issues
CallStack (from HasCallStack):
  error, called at src/Language/PureScript/Crash.hs:24:3 in purescript-0.11.6-E9oYDASU5kVKQAjU
68tzan:Language.PureScript.Crash
  internalError, called at src/Language/PureScript/CodeGen/JS.hs:128:35 in purescript-0.11.6-E
9oYDASU5kVKQAjU68tzan:Language.PureScript.CodeGen.JS

The cause is that the eval function will evaluate Var{} expressions. When evaluating Var{} expressions this should be limited to only those that points to number, character or boolean literal, but preserve string (because it might be repeated many times), array and object literals (this is the direct cause of the failure: they may contain references to other modules).

Breaks after upgrading to PureScript 0.15

Hey there. After updating to PureScript 0.15 zephyr fails with

$ zephyr -f Main.main
Error: Failed parsing:
  output/Control.Comonad.Traced/corefn.json
        output/Control.Monad.State/corefn.json
        output/Control.Monad.Error.Class/corefn.json
        output/Control.Monad.Identity.Trans/corefn.json
        output/Main/corefn.json
        ... (346 more)
error Command failed with exit code 1.

This is the offending branch https://github.com/dnikolovv/purescriptify/tree/purescript-0.15. You can run yarn run bundle to reproduce.

An NPM installer

Being able to install via NPM would probably be nice and lower the barrier to using zephyr

Binary fails with missing libtinfo.so.6

Downloading the binary for zephyr 0.3.2 fails on my Arch Linux system with the following error.

/zephyr: error while loading shared libraries: libtinfo.so.5: cannot open shared object file: No such file or directory

I worked around this issue by simply symlinking my libtinfo.so.5 library to libtinfo.so.6.

sudo ln -s /usr/lib/libtinfo.so.6 usr/liblibtinfo.so.5

I don't know if this work-around will go belly-up later (I haven't tested it), but I thought I'd pass it along in case it is helpful.

Build fails on macOS

Haskell and PureScript newbie here; I ran into this problem when trying to build purescript-halogen-realworld, which failed due to #38. Any insights are greatly appreciated!

  • macOS 10.15.5 (Catalina)
  • cabal 3.2.0.0 (via homebrew)
  • ghc 8.10.1 (via homebrew)
cabal build exe:zephyr   
Warning: Requested index-state2020-06-09T00:00:00Z is newer than
'hackage.haskell.org'! Falling back to older state (2020-06-08T22:11:02Z).
Resolving dependencies...
cabal: Could not resolve dependencies:
[__0] trying: zephyr-0.3.0 (user goal)
[__1] trying: purescript-0.13.8 (dependency of zephyr)
[__2] next goal: base (dependency of zephyr)
[__2] rejecting: base-4.14.0.0/installed-4.14.0.0 (conflict: purescript =>
base>=4.11 && <4.13)
[__2] skipping: base-4.14.0.0, base-4.13.0.0 (has the same characteristics
that caused the previous version to fail: excluded by constraint '>=4.11 &&
<4.13' from 'purescript')
[__2] rejecting: base-4.12.0.0, base-4.11.1.0, base-4.11.0.0, base-4.10.1.0,
base-4.10.0.0, base-4.9.1.0, base-4.9.0.0, base-4.8.2.0, base-4.8.1.0,
base-4.8.0.0, base-4.7.0.2, base-4.7.0.1, base-4.7.0.0, base-4.6.0.1,
base-4.6.0.0, base-4.5.1.0, base-4.5.0.0, base-4.4.1.0, base-4.4.0.0,
base-4.3.1.0, base-4.3.0.0, base-4.2.0.2, base-4.2.0.1, base-4.2.0.0,
base-4.1.0.0, base-4.0.0.0, base-3.0.3.2, base-3.0.3.1 (constraint from
non-upgradeable package requires installed instance)
[__2] fail (backjumping, conflict set: base, purescript, zephyr)
After searching the rest of the dependency tree exhaustively, these were the
goals I've had most trouble fulfilling: purescript, zephyr, base

For the record, a workaround is fixing the purescript version to 0.13.6 in purescript-halogen-realworld/package.json:

"purescript": "0.13.6"

Mention glibc requirement

zephyr binary isn't compatible with musl and requires glibc (what about uclibc?). The errors I was getting trying to use Alpine Linux while trying to use the v0.3.2 release binary wasn't very clear but eventually I tracked it down to this incompatibility. It could be useful to mention this somewhere in the docs.

Issue new release for Zephyr built with PureScript 0.13.x

At work we use Zephyr built from source with PureScript 0.13.3 (after #21 landed) and it works fine with files compiled with other PureScript 0.13.x versions.

However, I recently switched us over to use the Zephyr which is included in easy-purescript-nix, which is the last release: 0.2.1. Running this version on a project compiled with PureScript 0.13.6 on Linux causes this error:

zephyr: dce-output/Web.Fetch.Integrity/externs.json: openBinaryFile: does not exist (No such file or directory)

I put together a reproduction of most of our PureScript dependencies, which uses version 0.13.6 of the compiler and version 0.2.1 of Zephyr. But this reproduction fails when compiled and run on Mac OS with a different error:

zephyr: output/Web.HTML.HTMLTableHeaderCellElement/corefn.json: openBinaryFile: resource exhausted (Too many open files)

This is despite setting a large upper limit on file handles on Mac OS, as described in #9.

I think these errors are red herrings and that projects running on 0.13.x ought to use versions of Zephyr compiled with PureScript 0.13.x. I have been having trouble updating the reproduction to compile Zephyr from source on the master branch using PureScript 0.13.3, but I'll test that out tomorrow and make an update here.

That said: I believe it's worth making a new release compiled with PureScript 0.13.3 which could be added to easy-purescript-nix for everyone to use, regardless of whether I can fully reproduce this issue outside our codebase here.

Test on purescript-contrib libraries

purs compile bower_components/purescript-*/src/**/*.purs src/**/*.purs test/**/*test && zephyr -O1 Test.Main.main && node -e "require('./dce-output/Test.Main').main()"

[x] purescript-argonaut
[x] purescript-argonaut-core
[x] purescript-profunctor-lenses
[x] pulp (test suite & pulp itself)

Doesnt work with latest purescript

So it seems like the latest purescript version doesn't generate json externs anymore so running zephyr breaks with errors like:

zephyr: output/Affjax/externs.json: openBinaryFile: does not exist (No such file or directory)

Error Failed parsing: output/

I thought I would try out Zephyr and see if I could improve my build sizes, but I seem to be running into a problem when I run zephyr Main.main. Here's what I did.

First I checked out the zephyr repo and ran stack install, which added zephyr to my path. I have Stack Version 1.5.1 x86_64 hpack-0.17.1, PureScript 0.11.6, and Pulp 11.0.0. Then I went to my project's root and ran pulp build -- --dump-corefn which compiled my project. Then I ran zephyr Main.main and I got this error...

Error
Failed parsing:
  output/DOM.File.FileReader/corefn.json
        output/DOM.Event.WheelEvent/corefn.json
        output/Data.HeytingAlgebra/corefn.json
        output/DOM.HTML.HTMLMediaElement.NetworkState/corefn.json
        output/Control.Monad.Writer.Class/corefn.json
        output/Data.Bifunctor.Clown/corefn.json
        output/DOM.HTML.HTMLOptGroupElement/corefn.json
        output/Data.List.Lazy.NonEmpty/corefn.json
        output/Data.Distributive/corefn.json
        output/Data.Interval/corefn.json
        output/Control.Monad.Gen/corefn.json
        output/Data.Int.Bits/corefn.json
        output/Data.Monoid.Dual/corefn.json
        output/Data.Time.Component.Gen/corefn.json
        output/Data.Identity/corefn.json
        output/Data.Monoid/corefn.json
        output/DOM.HTML.HTMLTableElement/corefn.json
        output/Data.Char/corefn.json
        output/DOM.XHR.Types/corefn.json
        output/DOM.Node.NonElementParentNode/corefn.json
        output/Data.Traversable/corefn.json
        output/Control.Comonad.Store/corefn.json
        ....

I don't think there's anything unusual about the project I'm compiling. This is the bower.json file with my project dependencies.

Not compatible with PureScript 0.13.8

PureScript's 0.13.8 release changed the format of externs to use a binary encoding PR here.

If you try to use Zephyr on a project compiled with 0.13.8 you'll see an error like this:

zephyr: output/Affjax/externs.json: openBinaryFile: does not exist (No such file or directory)

There are no longer externs.json files. Instead, this would be a externs.cbor file in the same location in the output directory.

We'll need to update Zephyr to accommodate the new format.

(As a related note, there has been some discussion (1, 2) about moving Zephyr into the compiler, which would help with staying up-to-date with big changes in the compiler. Do you have any interest in this?)

file caching and dependency listings

Thank you very much for this nice tool. I use it for purescript-native cpp code generation:
PureScript Shared Objects
It is essential in the sence that the ffi implementations are not complete and zephyr allowes me to generate only the code which is just needed and does not pull in references of unimplemented functions. One issue i stumbled across is that zephyr creates new files for each run, regardless if the contents has changed or not. This will trigger all depending compilations which should not necessary at all. I worked around this by comparing the file sizes with the previous run, if they match then the new file gets the timestamp of the older file. (zephyr.bash in the mentioned repo)
The second issue is that I would like to get the dependencies for groups of entry points. I use this for automatic assembling of the libraries and applications. I worked around this by calling zephyr for each group for different output directories. This should be possible in one run too.
If this makes sense for you I would like to hear from you. I could try to implement this if you give me some guidance. Thanks a lot!

dce case expressions

It would be nice to remove dead branches of case expressions.

This is important for if statements:

if (Utils.production == "production")
  then ...
  else ...

For this case there could be a flag to provide value of Utils.production so zypher could infer that this is a production build.

macOS open file limits

I tried running zephyr on macOS 10.13 Beta (17A362a) in a clone of the pux-starter-app and got this:

± zephyr -f Main.main 
zephyr: output/Data.DateTime/corefn.json: openBinaryFile: resource exhausted (Too many open files)

You can work around the problem with these commands:

sudo sysctl -w kern.maxfiles=1000000
ulimit -S -n 8000

Sometimes produces more code than just bundle-app alone

I was surprised to see that using zephyr can sometimes increase bundle sizes. Is there a mistake in my workflow?

index.js parcel minified
Just bundle-app 316K 128K
With zephyr 332K 132K
git clone https://github.com/purescript-halogen/purescript-halogen-template.git bundle-test
cd bundle-test

# Workflow with just spago bundle-app and parcel (no zephyr)

npm run build-prod
du -hs prod/index.js dist/prod.*.js
# 316K    prod/index.js          # bundle-app output
# 128K    dist/prod.0a4f58ae.js  # minified by parcel

# Including zephyr

rm -r dist
spago build --purs-args '--codegen js,corefn' && zephyr Main.main --dce-foreign
purs bundle -o prod/index.js -m Main dce-output/**/*.js
parcel build prod/index.html
du -hs prod/index.js dist/prod.*.js
# 332K    prod/index.js          # zephyr + bundle-app output
# 132K    dist/prod.08b89072.js  # minified by parcel

https://discourse.purescript.org/t/recommended-tooling-for-purescript-in-2020/1615/4

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.