Giter Club home page Giter Club logo

split-require's Introduction

split-require

Bundle splitting for CommonJS and ES modules (dynamic import()) in browserify

Lazy load the parts of your app that are not immediately used, to make the initial load faster.

This module works without a compile step on the server, and in the browser with the browserify plugin.

NOTE: split-require v3+ works with browserify 16 and newer. If you are using an older browserify version and can't upgrade, use split-require v2.

What? - Install - Usage - Plugin CLI - Plugin API - License: MIT

stability travis standard

What?

This plugin takes source files with splitRequire() calls like:

var html = require('choo/html')
var app = require('choo')()
var splitRequire = require('split-require')

app.route('/', mainView)
app.mount('body')

var component
function mainView () {
  return html`<body>
    ${component ? component.render() : html`
      <h1>Loading</h1>
    `}
  </body>`
}

splitRequire('./SomeComponent', function (err, SomeComponent) {
  component = new SomeComponent()
  app.emitter.emit('render')
})

And splits off splitRequire()d files into their own bundles, that will be dynamically loaded at runtime. In this case, a main bundle would be created including choo, choo/html and the above file, and a second bundle would be created for the SomeComponent.js file and its dependencies.

Install

npm install split-require

Usage

Use the split-require function for modules that should be split off into a separate bundle.

var splitRequire = require('split-require')

require('./something') // loaded immediately
splitRequire('./other', function () {}) // loaded async

This works out of the box in Node.js. Add the browserify plugin as described below in order to make it work in the browser, too.

import()

You can use split-require with ES modules import() syntax using the Babel plugin.

browserify app.js \
  -t [ babelify --plugins dynamic-import-split-require ] \
  -p split-require
import('./SomeComponent').then(function (SomeComponent) {
  // Works!
  console.log(SomeComponent)
})

! Note that this transform is not 100% spec compliant. import() is an ES modules feature, not a CommonJS one. In CommonJS with this plugin, the exports object is the value of the module.exports of the imported module, so it may well be a function or some other non-object value. In ES modules, the exports object in .then() will always be an object, with a .default property for default exports and other properties for named exports. You'd never get a function back in .then() in native ES modules.

Browserify Plugin CLI Usage

browserify ./entry -p [ split-require --out /output/directory ]
  > /output/directory/bundle.js

Options

--out

Set the output for dynamic bundles. Use a folder path to place dynamic bundles in that folder. You can also use outpipe syntax: in that case use %f in place of the bundle name. For example:

-p [ split-require --out 'uglifyjs > /output/directory/%f' ]

The default is ./, outputting in the current working directory.

--public

Public path to load dynamic bundles from. Defaults to ./, so dynamic bundle #1 is loaded as ./bundle.1.js.

--sri

Hash algorithm to use for subresource integrity. By default, subresource integrity is not used.

When enabled, the runtime loader will add crossorigin and integrity attributes to dynamically loaded bundles.

One of sha256, sha384, sha512.

Browserify Plugin API Usage

var splitRequire = require('split-require')

browserify('./entry')
  .plugin(splitRequire, {
    dir: '/output/directory'
  })
  .pipe(fs.createWriteStream('/output/directory/bundle.js'))

With factor-bundle

Through the API, split-require can also be used together with factor-bundle. Listen for the factor.pipeline event, and unshift the result of the createStream function to the 'pack' label:

b.plugin(splitRequire, { dir: '/output/directory' })
b.plugin(factorBundle, { /* opts */ })

b.on('factor.pipeline', function (file, pipeline) {
  var stream = splitRequire.createStream(b, {
    dir: '/output/directory'
  })
  pipeline.get('pack').unshift(stream)
})

Note that you must pass the options to the plugin and the stream. Other plugins that generate multiple outputs may need a similar treatment.

Options

dir

Set the folder to output dynamic bundles to. Defaults to ./. This is only necessary if the output() option is not used.

filename(entry)

Function that generates a name for a dynamic bundle. Receives the entry point row as the only parameter. The default is:

function filename (entry) {
  return 'bundle.' + entry.index + '.js'
}

output(bundleName)

Function that should return a stream. The dynamic bundle will be written to the stream. bundleName is the generated name for the dynamic bundle. At runtime, the main bundle will attempt to use this name to load the bundle, so it should be publically accessible under this name.

The bundleName can be changed by emitting a name event on the returned stream before the stream finishes. This is useful to generate a bundle name based on a hash of the file contents, for example.

var fs = require('fs')
var crypto = require('crypto')
var to = require('flush-write-stream')
b.plugin(splitRequire, {
  output: function (bundleName) {
    var stream = fs.createWriteStream('/tmp/' + bundleName)
    var hash = crypto.createHash('sha1')
    return to(onwrite, onend)

    function onwrite (chunk, enc, cb) {
      hash.update(chunk)
      stream.write(chunk, cb)
    }
    function onend (cb) {
      stream.end()
      var name = hash.digest('hex').slice(0, 10) + '.js'
      this.emit('name', name)
      fs.rename('/tmp/' + bundleName, './' + name, cb)
    }
  }
})

public

Public path to load dynamic bundles from. Defaults to ./, so dynamic bundle #1 is loaded as ./bundle.1.js.

sri

Hash algorithm to use for subresource integrity. By default, subresource integrity is not used.

When enabled, the runtime loader will add crossorigin and integrity attributes to dynamically loaded bundles.

One of sha256, sha384, sha512.

Events

b.on('split.pipeline', function (pipeline, entry, basename) {})

split-require emits an event on the browserify instance for each pipeline it creates.

pipeline is a labeled-stream-splicer with labels:

entry is the browserify row object for the entry point of the dynamic bundle. basename is the name of the dynamic bundle file.

License

MIT

split-require's People

Contributors

dependabot-preview[bot] avatar goto-bus-stop avatar sylvainpolletvillard avatar zifeo avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

Forkers

scoutgg

split-require's Issues

play nice with factor-bundle

Should unshift this plugin to the 'pack' label, and do the same on 'factor.pipeline' event, I think we're already good then.

Error bundling dynamic import + JSX

I'm having trouble getting split-require with a ES2015 + JSX project to work (cf. minimal example project). It fails bundling dynamic import + JSX code.

I was not able to track down the issue to the babel transform for dynamic import or split-require. Both seem to work fine alone but fail when being combined.

Test

import('./external').then(function(result) {
  console.log(result);
});


var app = <div></div>;

Error

❯ npm test
> @ test ~/Projects/split-require-jsx-text
> browserify index.js -t [ babelify ] -p [ split-require --dir dist ] -o dist/index.js
~/Projects/split-require-jsx-text/node_modules/split-require/plugin.js:327
    var requirePath = node.arguments[0].arguments[0].value
                                                 ^
TypeError: Cannot read property '0' of undefined
    at processSplitRequire (~/Projects/split-require-jsx-text/node_modules/split-require/plugin.js:327:50)
    at ~/Projects/split-require-jsx-text/node_modules/split-require/plugin.js:144:11
    at walk (~/Projects/split-require-jsx-text/node_modules/transform-ast/index.js:82:5)
    at walk (~/Projects/split-require-jsx-text/node_modules/transform-ast/index.js:78:9)
    at walk (~/Projects/split-require-jsx-text/node_modules/transform-ast/index.js:74:13)
    at walk (~/Projects/split-require-jsx-text/node_modules/transform-ast/index.js:78:9)
    at walk (~/Projects/split-require-jsx-text/node_modules/transform-ast/index.js:74:13)
    at walk (~/Projects/split-require-jsx-text/node_modules/transform-ast/index.js:78:9)
    at walk (~/Projects/split-require-jsx-text/node_modules/transform-ast/index.js:78:9)
    at walk (~/Projects/split-require-jsx-text/node_modules/transform-ast/index.js:78:9)
npm ERR! Test failed.  See above for more details.

split-require returns empty object when minfied

I'm seeing some really strange behavior when using split-require together with tinyify. I am trying to conditionally load a large library for browser that don't support window.crypto.subtle like this:

var splitRequire = require('split-require')

module.exports = bindCrypto

var cryptoProvider = window.crypto && window.crypto.subtle
  ? getWebCrypto()
  : getForgeCrypto()

function bindCrypto (consumerFn) {
  return function () {
    var args = [].slice.call(arguments)
    return cryptoProvider
      .then(function (cryptoImplementation) {
        return consumerFn.apply(cryptoImplementation, args)
      })
  }
}

function getWebCrypto () {
  return Promise.resolve(require('./web-crypto'))
}

function getForgeCrypto () {
  return new Promise(function (resolve, reject) {
    splitRequire('./forge-crypto', function (err, forgeCrypto) {
      if (err) {
        return reject(err)
      }
      resolve(forgeCrypto)
    })
  })
}

This works as expected, but strangely I noticed that some time early September 2020 it looks like the splitRequire call passes an empty object ({}) to the function instead of the module's export.

However this only happens when I apply tinyify to the code above. If I leave it unminified, I will be passed the correct export. It does not seem to matter if the chunk that is split off is minified or not.

Quite frankly, this does not make any sense to me considering that it used to work and broke without any intervention (I am aware of) apparently, so I would be happy if anyone has any hint about how to even debug this?

For the sake of completeness this is how I bundle things:

var transforms = JSON.parse(JSON.stringify(pkg.browserify.transform))
var b = browserify({
  entries: './index.js',
  // See: https://github.com/nikku/karma-browserify/issues/130#issuecomment-120036815
  postFilter: function (id, file, currentPkg) {
    if (currentPkg.name === pkg.name) {
      currentPkg.browserify.transform = []
    }
    return true
  },
  transform: transforms.map(function (transform) {
    if (transform === '@offen/l10nify') {
      return ['@offen/l10nify', { locale: locale }]
    }
    return transform
  })
})

b.on('split.pipeline', function (pipeline) {
  tinyify.applyToPipeline(pipeline)
})

return b
  .exclude('dexie')
  .plugin('tinyify') // commenting out this line makes things work, but I end up with non-minified code
  .plugin('split-require', {
    dir: dest,
    filename: function (entry) {
      return 'chunk-' + entry.index + '.js'
    },
    sri: 'sha384',
    output: function (bundleName) {
      var buf = ''
      return to(onwrite, onend)

      function onwrite (chunk, enc, cb) {
        buf += chunk
        cb()
      }

      function onend (cb) {
        var hash = crypto.createHash('sha1').update(buf)
        var name = bundleName.replace(/\.js$/, '') + '-' + hash.digest('hex').slice(0, 10) + '.js'
        this.emit('name', name)
        fs.writeFile(dest + name, buf, cb)
      }
    }
  })
  .bundle()
  .pipe(source('index.js'))
  .pipe(buffer())
  .pipe(gap.prependText('*/'))
  .pipe(gap.prependFile('./../banner.txt'))
  .pipe(gap.prependText('/**'))
  .pipe(rev())
  .pipe(gulp.dest(dest))
  .pipe(rev.manifest(dest + 'rev-manifest.json', { base: dest, merge: true }))
  .pipe(gulp.dest(dest))

--out CLI option does not work

I tried to use the --out CLI option as described in the README: https://github.com/goto-bus-stop/split-require#--out

browserify app/index.js -p [ split-require --out dist ]
browserify app/index.js -p [ split-require --out ./dist/ ]

Nothing worked. Also I checked the source and cannot find the out option being used. The only option In found is dir:

var outputDir = opts.dir || './'

I tried with the --dir option and it worked !

browserify app/index.js -p [ split-require --dir ./dist/ ]

Do you need to update the docs ?

Script not found (url is undefined)

Hi, I'm trying to setup browserify+babel+split-require, but I keep receiving an undefined request:
cap

As far as I have checked, load.b is a empty object, this is why url is undefined.

var url = load.b[index]

Here is my code

document.addEventListener('DOMContentLoaded', () => {
  splitRequire('./test', function() {
    console.log(arguments);
  });
});

And Here is how the browserify script is setup:
browserify ./source/js/index.js -t [ babelify --plugins dynamic-import-split-require ] -p [split-require --out './dist/js/%f'] > ./dist/js/bundle.js

I've checked the output files and seems ok:

bundle.js
bundle.3.js

Here is the splitRequire on bundle.js

document.addEventListener('DOMContentLoaded', function () {
  splitRequire(3, function () {
    console.log(arguments);
  });
});

Compiled bundle.js: https://pastebin.com/B2WUKe3n

Could someone help me : ) ?
Thanks

[Question] Is there any way we could use this and uglify the generated bundles?

I'm new to Browserify and trying to use this to code split some routes. So far everything works as expected (I'm using the package.json scripts)

"scripts": {
    ...
    "build:js": "NODE_ENV=production browserify index.js -p [ split-require --dir static/ --public static/ ] | uglifyjs -c > static/bundle.js",
    "build": "npm run build:css && npm run build:js",
    ...
  },

There's only one thing I noticed: the generated bundles aren't uglified (of course, I'm just uglifying the result of browserify index.js).

Is there a way to uglify the results of this plugin?

option to output to a stream instead of a file

can just do pipeline.pipe(outputStream(bundle)) instead of pipeline.pipe(fs.createWriteStream(outputName(bundle))).

one thing to consider is the mapping of entry point IDs to publically accessible URLs, maybe will have to generate the dynamic bundle names up front again so the mapping can be added into the main bundle in time. may have to remove the async outname option, which can be used to generate hash-based filenames for dynamic bundles, but i think that's probably ok and it was clumsy to use anyway.

capture required bundles during server rendering

so you can generate script tags for everything that is needed to render the page.

maybe having an api like splitRequire.capture(function (cb) { render(cb) }, function (result, bundles) {}) could be cool; could use the new async_hooks probably.

Returning promises (optionally)

I'm building a PWA with Choo using asynchronous components and given that I'm using promises to deal with those it would be interesting if split-require could return a promise rather that using callbacks only. Something like:

const splitRequire = require('split-require')

splitRequire('./SomeComponent').then(SomeComponent => { ... }) // promise
splitRequire('./SomeComponent', (err, SomeComponent) => { ... }) // callback

tests using tape-run

instead of comparing bundle outputs, run tests in a browser and check that things actually work.

bundle outputs are annoying to update, if the runtime changes a single line every test needs to change :(

subresource integrity

need to set .crossOrigin and .integrity properties on the script tags in the browser runtime.

--full-paths

split-require assumes indexDeps exist on files, but it looks like they don't when browserify is run with --full-paths.

split-require and factor-bundle chicken/egg

Hi!

I'm a little confused by the combination of split-require and factor-bundle in tandem.

Is it possible for split-require to emit all files so they can be passed to factor-bundle ahead of time?
Looking at the events it seems that split-require already leaves out dependencies from the main entry from the dynamic imports, but is it possible somehow to use factor bundle to take out all external dependencies?

In my test it seems factor.pipeline emits before split.pipeline, so I can't inject factor bundle later?

So my main entry file will contain a lot of shared dependencies, but a subset of my dynamic imports will also share dependencies. I'm trying to get a setup with hash based file naming so that all files are renamed to their hash and I can serve them with Cache-Control: Immutable, where npm dependencies will likely change very rarely, while my source files will sometimes change.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.