Giter Club home page Giter Club logo

webpack-node-externals's Introduction

Webpack node modules externals

Easily exclude node modules in Webpack

Version Downloads Build Status

Webpack allows you to define externals - modules that should not be bundled.

When bundling with Webpack for the backend - you usually don't want to bundle its node_modules dependencies. This library creates an externals function that ignores node_modules when bundling in Webpack.
(Inspired by the great Backend apps with Webpack series)

Quick usage

npm install webpack-node-externals --save-dev

In your webpack.config.js:

const nodeExternals = require('webpack-node-externals');
...
module.exports = {
    ...
    target: 'node', // in order to ignore built-in modules like path, fs, etc.
    externals: [nodeExternals()], // in order to ignore all modules in node_modules folder
    ...
};

And that's it. All node modules will no longer be bundled but will be left as require('module').

Note: For Webpack 5, in addition to target: 'node' also include the externalsPreset object:

// Webpack 5

const nodeExternals = require('webpack-node-externals');
...
module.exports = {
    ...
    target: 'node', 
    externalsPresets: { node: true }, // in order to ignore built-in modules like path, fs, etc.
    externals: [nodeExternals()], // in order to ignore all modules in node_modules folder
    ...
};

Detailed overview

Description

This library scans the node_modules folder for all node_modules names, and builds an externals function that tells Webpack not to bundle those modules, or any sub-modules of theirs.

Configuration

This library accepts an options object.

options.allowlist (=[])

An array for the externals to allow, so they will be included in the bundle. Can accept exact strings ('module_name'), regex patterns (/^module_name/), or a function that accepts the module name and returns whether it should be included.
Important - if you have set aliases in your webpack config with the exact same names as modules in node_modules, you need to allowlist them so Webpack will know they should be bundled.

options.importType (='commonjs')

The method in which unbundled modules will be required in the code. Best to leave as commonjs for node modules. May be one of documented options or function callback(moduleName) which returns custom code to be returned as import type, e.g:

options.importType = function (moduleName) {
    return 'amd ' + moduleName;
}

options.modulesDir (='node_modules')

The folder in which to search for the node modules.

options.additionalModuleDirs (='[]')

Additional folders to look for node modules.

options.modulesFromFile (=false)

Read the modules from the package.json file instead of the node_modules folder.
Accepts a boolean or a configuration object:

{
    modulesFromFile: true,
    /* or */
    modulesFromFile: {
        fileName: /* path to package.json to read from */,
        includeInBundle: [/* whole sections to include in the bundle, i.e 'devDependencies' */],
        excludeFromBundle: [/* whole sections to explicitly exclude from the bundle, i.e only 'dependencies' */]
    }
}

Usage example

var nodeExternals = require('webpack-node-externals');
...
module.exports = {
    ...
    target: 'node', // important in order not to bundle built-in modules like path, fs, etc.
    externals: [nodeExternals({
        // this WILL include `jquery` and `webpack/hot/dev-server` in the bundle, as well as `lodash/*`
        allowlist: ['jquery', 'webpack/hot/dev-server', /^lodash/]
    })],
    ...
};

For most use cases, the defaults of importType and modulesDir should be used.

Q&A

Why not just use a regex in the Webpack config?

Webpack allows inserting regex in the externals array, to capture non-relative modules:

{
    externals: [
        // Every non-relative module is external
        // abc -> require("abc")
        /^[a-z\-0-9]+$/
    ]
}

However, this will leave unbundled all non-relative requires, so it does not account for aliases that may be defined in webpack itself. This library scans the node_modules folder, so it only leaves unbundled the actual node modules that are being used.

How can I bundle required assets (i.e css files) from node_modules?

Using the allowlist option, this is possible. We can simply tell Webpack to bundle all files with extensions that are not js/jsx/json, using this regex:

...
nodeExternals({
  // load non-javascript files with extensions, presumably via loaders
  allowlist: [/\.(?!(?:jsx?|json)$).{1,5}$/i],
}),
...

Thanks @wmertens for this idea.

Why is not bundling node_modules a good thing?

When writing a node library, for instance, you may want to split your code to several files, and use Webpack to bundle them. However - you wouldn't want to bundle your code with its entire node_modules dependencies, for two reasons:

  1. It will bloat your library on npm.
  2. It goes against the entire npm dependencies management. If you're using Lodash, and the consumer of your library also has the same Lodash dependency, npm makes sure that it will be added only once. But bundling Lodash in your library will actually make it included twice, since npm is no longer managing this dependency.

As a consumer of a library, I want the library code to include only its logic, and just state its dependencies so they could me merged/resolved with the rest of the dependencies in my project. Bundling your code with your dependencies makes it virtually impossible.

In short: It's useful if your code is used by something that has dependencies managed by npm

Contribute

Contributions and pull requests are welcome. Please run the tests to make sure nothing breaks.

Test

npm run test

License

MIT

webpack-node-externals's People

Contributors

brandon-leapyear avatar dependabot[bot] avatar liady avatar mcous avatar ndelangen avatar pwmckenna avatar snaerth avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

webpack-node-externals's Issues

correct configuration for files that do not exist in development

I have the following code:

    const configureProduction = (app: any) => {
      const clientStats = require('./stats.json');
      const serverRender = require('./server.js').default;
      const publicPath = '/';
      const outputPath = join(__dirname, '.');
    
      app.use(publicPath, express.static(outputPath));
      app.use(
        serverRender({
          clientStats,
          outputPath
        })
      );
    
      app.set('views', join(__dirname, 'views'));
    };
    
    const app = express();
    
    if (process.env.NODE_ENV === 'development') {
      configureDevelopment(app);
    } else {
      configureProduction(app);
    }

When running in development, the configureProduction function is not called but I get the following webpack errors:

ERROR in ./src/index.tsx Module not found: Error: Can't resolve
'./server.js' in
'/Users/paulcowan/projects/cuttingedge/packages/website/src' @
./src/index.tsx 38:23-45 @ multi ./src/index

ERROR in ./src/index.tsx Module not found: Error: Can't resolve
'./stats.json' in
'/Users/paulcowan/projects/cuttingedge/packages/website/src' @
./src/index.tsx 37:22-45 @ multi ./src/index

My nodeExternals is set like this:

    externals: [
      nodeExternals({
        modulesDir: path.join(process.cwd(), '../../node_modules'),
        whitelist: [
          isDevelopment ? 'webpack/hot/poll?300' : null,
          './stats.json',
          './server.js',
          /\.(eot|woff|woff2|ttf|otf)$/,
          /\.(svg|png|jpg|jpeg|gif|ico)$/,
          /\.(mp4|mp3|ogg|swf|webp)$/,
          /\.(css|scss|sass|sss|less)$/
        ].filter(x => x)
      })
    ],

How can I configure this correctly?

1.3.0 update - cannot find module

After updating from 1.2.0 to 1.3.0 I am getting the following error:

Error: Cannot find module '/Users/chris/app/node_modules/extract-text-webpack-plugin/loader.js?{"omit":1,"extract":true,"remove":true}!style!css!postcss!resolve-url!sass?sourceMap!./lib/bootstrap.styles.loader.js!./no-op.js'
    at Function.Module._resolveFilename (module.js:325:15)
    at Function.Module._load (module.js:276:25)
    at Module.require (module.js:353:17)
    at require (internal/module.js:12:17)
    at Object.<anonymous> (webpack:///./no-op.js":1:1)
    at __webpack_require__ (webpack:///webpack/bootstrap ffe52b1d2cc71d90e522:19:1)
    at Object.module.exports.module.exports.forcePointer (webpack:///./~/bootstrap-loader/no-op.js:1:1)
    at __webpack_require__ (webpack:///webpack/bootstrap ffe52b1d2cc71d90e522:19:1)
    at Object.<anonymous> (webpack:///./~/bootstrap-loader/extractStyles.js:1:1)
    at __webpack_require__ (webpack:///webpack/bootstrap ffe52b1d2cc71d90e522:19:1)

Ideas? Thank you!

Module not found: Error: Can't resolve 'hiredis'

I use nodeExternals a lot to build server-side modules, but I get this error for any project that depends (indirectly) on redis (all other modules -- 1000s of them -- work fine).

In this case, my import of bull causes problems, through its dependency on ioredis (similar issues webpack/webpack-dev-server#227 and redis/node-redis#790)

Solution may relate to #29

ERROR in ../sub/scheduler/~/bull/~/redis-parser/lib/hiredis.js
Module not found: Error: Can't resolve 'hiredis' in '/.../scheduler/node_modules/bull/node_modules/redis-parser/lib'
 @ ../scheduler/~/bull/~/redis-parser/lib/hiredis.js 3:14-32
 @ ../scheduler/~/bull/~/redis-parser/lib/parser.js
 @ ../scheduler/~/bull/~/redis-parser/index.js
 @ ../scheduler/~/bull/~/ioredis/lib/redis/parser.js
 @ ../scheduler/~/bull/~/ioredis/lib/redis.js
 @ ../scheduler/~/bull/~/ioredis/index.js
 @ ../scheduler/~/bull/lib/queue.js
 @ ../scheduler/src/util/queue.js
 @ ../scheduler/index.js
 @ ./src/server/router/admin.js
 @ ./src/server/server.js
 @ ./src/server/main.js
 @ multi babel-polyfill ./src/server/main.js

The workaround is to add to "optionalDependencies" all modules that my other sub projects depend upon and set modulesFromFile but this is messy.

Peer dependency of required module is not bundled

Hi, I use request-promise package which has peer dependency on request but only request-promise is bundled and request is missing. I used default nodeExternals() setup and switched to nodeExternals({ modulesFromFile: true }) but it is still the same (althoug I have both request and request-promise in dependencies of my package.json).

So I have to require the request in my code even if I don't use it directly. Have I forgotten to setup something or it is a bug?

whitelisted files may point to wrong node_modules

say you whitelist module 'a', which has a dependency on module 'b'.

node_modules
  a
    node_modules
      b

whitelisting a will pull it into the root directory, at which point require('b') statements in a will start to fail because they now look in the wrong node_modules directory. whitelisted modules may need their require statements rewired to point to absolute paths, or at least new relative paths.

Here's some similar code that I use to rewire require statements inside of a webpack externals function.

webpack.config.js

var resolve = require('resolve');
module.exports = {
    ...
    externals: [function (context, request, callback) {
        var absolute = resolve.sync(request, {
            basedir: context,
            extensions: ['.js', '.json']
        });
        callback(null, 'commonjs ' + absolute);
    }]
};

css files from node_modules are still being included

I'm using this plugin as such:

externals: [
      nodeExternals(),
    ],

In one of my sass files, I do

@import '~normalize/normalize.css'

The generated server bundle still contains the entire normalize.css as a string.

How to whitelist all .css imports?

Can't find a way to whitelist all imports that would require a .css files from node_modules
I tried with this regex, the same that my loader: /.scss|sass|css$/
but no luck...

node_modules are not excluded

webpack-cli version = 3.1.0

package.json

   {
  "name": "zinuku",
  "version": "1.0.0",
  "description": "",
  "main": "index.jsx",
  "scripts": {
    "build": "webpack-cli",
    "start": "webpack-dev-server"
  },
  "dependencies": {
    "react": "^16.4.1",
    "react-dom": "^16.4.1"
  },
  "devDependencies": {
    "babel-core": "^6.26.3",
    "babel-loader": "^7.1.5",
    "babel-preset-react": "^6.24.1",
    "html-webpack-plugin": "^3.2.0",
    "webpack": "^4.16.3",
    "webpack-cli": "^3.1.0",
    "webpack-dev-server": "^3.1.5",
    "webpack-node-externals": "^1.7.2"
  }
}

webpack.config.js

const webpack = require('webpack');
const nodeExternals = require('webpack-node-externals');

var HTMLWebpackPlugin = require('html-webpack-plugin');
var HTMLWebpackPluginConfig = new HTMLWebpackPlugin({
    template: __dirname + '/app/index.html',
    filename: 'index.html',
    inject: 'body'
});


module.exports = {
    entry: __dirname + '/app/index.jsx',
    mode: 'development',
    target: 'node', // in order too ignore built-in modules like path, fs, etc.
    externals: [nodeExternals()], // in order to ignore all modules in node_modules folder
    module: {
        rules: [
            {
                test: /\.js$/,
                exclude: /node_modules/,
                loader: 'babel-loader'
            },
            {
                test: /\.jsx$/,
                exclude: /node_modules/,
                loader: 'babel-loader'
            }
        ]
    },
    output: {
        filename: 'transformed.js',
        path:  __dirname + '/build'
    },
    plugins: [HTMLWebpackPluginConfig]
};

Bash (displaying that node_modules got build too!)

npm run start

> [email protected] start C:\Users\sebas\pycharmProjects\zinuku
> webpack-dev-server

i ๏ฝขwds๏ฝฃ: Project is running at http://localhost:8081/
i ๏ฝขwds๏ฝฃ: webpack output is served from /
i ๏ฝขwdm๏ฝฃ: Hash: 94790fc256e631986c59
Version: webpack 4.16.3
Time: 873ms
Built at: 2018-07-31 20:44:54
         Asset       Size  Chunks             Chunk Names
transformed.js     27 KiB    main  [emitted]  main
    index.html  221 bytes          [emitted]
Entrypoint main = transformed.js
[./app/components/App.jsx] 386 bytes {main} [built]
[./app/components/HeaderComponent/NavBar.jsx] 294 bytes {main} [built]
[./app/index.jsx] 209 bytes {main} [built]
[./node_modules/webpack-dev-server/client/index.js?http://localhost:8081] (webpack)-dev-server/client?http://localhost:8081 7.78 KiB {main} [built]
[./node_modules/webpack-dev-server/client/overlay.js] (webpack)-dev-server/client/overlay.js 3.58 KiB {main} [built]
[./node_modules/webpack-dev-server/client/socket.js] (webpack)-dev-server/client/socket.js 1.05 KiB {main} [built]
[./node_modules/webpack/hot sync ^\.\/log$] (webpack)/hot sync nonrecursive ^\.\/log$ 170 bytes {main} [built]
[loglevel] external "loglevel" 42 bytes {main} [built]
[react] external "react" 42 bytes {main} [built]
[react-dom] external "react-dom" 42 bytes {main} [built]
[sockjs-client/dist/sockjs] external "sockjs-client/dist/sockjs" 42 bytes {main} [built]
[strip-ansi] external "strip-ansi" 42 bytes {main} [built]
[url] external "url" 42 bytes {main} [built]
[webpack/hot/emitter] external "webpack/hot/emitter" 42 bytes {main} [built]
[0] multi (webpack)-dev-server/client?http://localhost:8081 ./app/index.jsx 40 bytes {main} [built]
    + 3 hidden modules
Child html-webpack-plugin for "index.html":
     1 asset
    Entrypoint undefined = index.html
    [./node_modules/html-webpack-plugin/lib/loader.js!./app/index.html] 372 bytes {0} [built]
    [./node_modules/lodash/lodash.js] 527 KiB {0} [built]
    [./node_modules/webpack/buildin/module.js] 497 bytes {0} [built]
i ๏ฝขwdm๏ฝฃ: Compiled successfully.

My file structure: https://gyazo.com/670cf8afbae2fb213d759d455221d7f3

Base paths on your code that share names with node modules get ignored

Hey, still trying to learn a little more about this module and mocha-webpack to get our team's testing suite a lot less janky, and it's looking very promising. Unfortunately I was getting a "unable to find module" error for something I know to work, and did a lot of head scratching for a while. Fortunately I finally realized that there may be an issue with the path it was at, util/.... Even though this path knows to resolve in our directory first, webpack-node-externals seems to be mistaking it for https://www.npmjs.com/package/util and subsequently ignoring it.

I'll update this issue as I get more details and hopefully a simple reproduction case going, or even better a fix.

TypeError: Path must be a string. Received undefined

This is my code, I use typescript, but having issue with alias, but it turns out that is because I use nodeExternals() that's why the alias but work, but after I add the externals: [nodeExternals({
whitelist: ['client-logger']
})]

I'm getting some other error

2018-09-10T02:29:16Z: identifier="routes:build-info" level="WARN" message="Build Info file not found, and hence not set up to inform actual version of the app." 
Warning: React depends on requestAnimationFrame. Make sure that you load a polyfill in older browsers. https://fb.me/react-polyfills
path.js:28
    throw new TypeError('Path must be a string. Received ' + inspect(path));
    ^

TypeError: Path must be a string. Received undefined
    at assertPath (path.js:28:11)
    at Object.dirname (path.js:1349:5)
    at Object.<anonymous> (/Users/stanley/Desktop/X/epc-photos-pwa-app/assets/server.js:1:928603)
    at t (/Users/stanley/Desktop/X/epc-photos-pwa-app/assets/server.js:1:419)
    at Object.<anonymous> (/Users/stanley/Desktop/X/epc-photos-pwa-app/assets/server.js:1:928105)
    at t (/Users/stanley/Desktop/X/epc-photos-pwa-app/assets/server.js:1:419)
    at Object.defineProperty.value (/Users/stanley/Desktop/X/epc-photos-pwa-app/assets/server.js:1:926987)
    at t (/Users/stanley/Desktop/X/epc-photos-pwa-app/assets/server.js:1:419)
    at Object.<anonymous> (/Users/stanley/Desktop/X/epc-photos-pwa-app/assets/server.js:1:854113)
    at t (/Users/stanley/Desktop/X/epc-photos-pwa-app/assets/server.js:1:419)
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] start: `node --use_strict ./bin/www`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the [email protected] start script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

Check this for more, plese help https://stackoverflow.com/questions/52201186/webpacks-alias-doesnt-work-when-target-module-is-in-node-modules

Doesn't work with yarn workspaces by default

I have a monorepo which is essentially a repository with many npm packages in it and I just started using yarn workspaces which hoists all dependencies to the root of the repository. I realized that all my node_modules were getting included because they were now located in a node_modules folder two parents up. So I had to switch to externals: /node_modules/

Allow whitelisting of a module/subpath

Hi, I have a use case:

// This module is whitelisted in a config
import 'some_module'

// This one won't be whitelisted and needs
// to be added explicitly to the whitelist
import 'some_module/subpath/to/some_file' 

Is there some way to resolve this issue?

transitive deps

I'm not sure how transitive deps work with this module. Say if a direct dep is whitelisted, are the transitive deps of the direct dep also whitelisted? Can you also whitelist a transitive dep?

[Question] do I need to add exclude section to loaders with this library?

Hi, I have a broad question about how this library works. If I use webpack-node-externals like so:

const nodeExternals = require('webpack-node-externals')
module.exports {
  ...
  externals: [nodeExternals()]
}

what does it actually do? The webpack documentation states

The externals configuration option provides a way of excluding dependencies from the output bundles

What I want to know, is if this also prevents anything in the node_modules folder from reaching the loaders. A lot of loaders recommend adding an exclude: /node_modules/ section to prevent node_modules from being transpiled (babel-loader, awesome-typescript-loader, istanbul-instrumenter-loader, etc). I want to know if this step is necessary provided we are already using webpack-node-externals.

Entry with webpack/hot/poll?1000 isn't correctly treated as an external

When I have something like:

{
  entry: ["webpack/hot/poll?1000", "./src/client.js"],
  externals: [nodeExternals()],
}

I get an error from Webpack like:

Module not found: webpack/hot/poll?1000

I tried forking the repo to demonstrate the problem, but had a difficult time working out the tests.

Let me know if this passes the sniff test for you, otherwise I'll do what I can to help.

Thanks!

ignore devDependencies

When I put a dependency into devDependencies without dependencies, I'm intending to indicate "this module only needs to be installed to build my library." My compiled library should not expect that the module is installed.

With the current behavior of webpack-node-externals, even if I specify modulesFromFile, any dependencies in devDependencies will not be bundled with my library, nor will they be installed by npm (unless the user explicitly opts-in to dev). The result is a bundle that works in my dev environment, passes tests in dev, and fails when pushed out to users!

I expect devDependencies to be part of the bundle, not marked as externals. If necessary (to avoid a breaking change), please make this a flag so I can opt-in to this behavior.

Still pulling in certain modules, namely React internals

Wondering if it's user error ๐Ÿ˜„

FWIW, in my context the regex works fine, as I'm not overriding module resolution.

/***/ },
/* 163 */
/*!*********************************************************!*\
  !*** ../~/react/lib/ReactServerRenderingTransaction.js ***!
  \*********************************************************/
/***/ function(module, exports, __webpack_require__) {

  /**
   * Copyright 2014-present, Facebook, Inc.
   * All rights reserved.
   *
   * This source code is licensed under the BSD-style license found in the
   * LICENSE file in the root directory of this source tree. An additional grant
   * of patent rights can be found in the PATENTS file in the same directory.
   *
   * @providesModule ReactServerRenderingTransaction
   */

  'use strict';

  var _assign = __webpack_require__(/*! object-assign */ 25);

  var PooledClass = __webpack_require__(/*! ./PooledClass */ 26);
  var Transaction = __webpack_require__(/*! ./Transaction */ 46);

config looks like this

// ...
    entry['app-server'] = './src/index'
    entry['app-worker'] = './src/worker'
// ...
  const config = {
    devtool: 'cheap-source-map',
    entry,
    output: {
      path: path.resolve('dist'),
      filename: '[name].js',
      publicPath: '/assets/'
    },
// ...
    config.target = 'node'
    config.output.libraryTarget = 'commonjs'
    config.externals = [externals()] // <-- webpack-node-externals
    config.node = { // webpack strangely doesn't do this when you set `target: 'node'`
      console: false,
      global: false,
      process: false,
      Buffer: false,
      __filename: false,
      __dirname: false,
      setImmediate: false
    }

Whitelist scoped modules?

Is it possible to whitelist scoped modules?

For instance:

{
	whitelist:
	[
		'@scope/module',
		/@scope\/.+/
	]
}

I attempted to use both of these formats, but unfortunately the relevant modules were not bundled.

Not doing anything?

I think I might just completely misunderstand what this does. I was hoping that webpack would not bundle everything I imported by using this (making my npm package about 300 kb vs 40). I originally just listed everything manually which worked for the most part but it was missing a few files in child folders of the packages.

So I added this and tried it and sure enough everything is now included again.

process-handle.js  242 kB       0  [emitted]  main
   [9] ./src/process-lib/helpers.js 1.16 kB {0} [built]
  [10] ./src/process-lib/wildcard.js 3.31 kB {0} [built]
  [23] ../~/redux-saga/effects.js 41 bytes {0} [built]
  [29] ./src/process-lib/createActions.js 2.22 kB {0} [built]
  [30] ./src/process-lib/registry.js 2.64 kB {0} [built]
  [31] ./src/process-lib/statics.js 521 bytes {0} [built]
  [32] ./src/main.js 2.12 kB {0} [built]
  [33] ./src/statics.js 373 bytes {0} [built]
  [46] ../~/react/react.js 56 bytes {0} [built]
  [50] ../~/redux-saga/utils.js 40 bytes {0} [built]
  [51] ../~/reselect/lib/index.js 3.83 kB {0} [built]
  [52] ./src/process-lib/effects.js 11.5 kB {0} [built]
  [53] ./src/process-lib/process.js 14 kB {0} [built]
  [54] ./src/process-lib/reducerGenerators.js 2.79 kB {0} [built]
  [55] multi ./src/statics.js ./src/main.js 40 bytes {0} [built]
    + 41 hidden modules
Done in 4.59s.

I just have this in my config

 externals: [ NodeExternals() ]

Node's optionalDependencies are not working

Hey,

This package is great! I am certainly not an expert with webpack config files but I've came across this issue where I had an optional package and using this, webpack just couldn't figure out that it is optional.
It relates to the change webpack did here webpack/webpack@5104661
It seems that webpack recognises optional packages when the externals object is directing to the global package name like so:

externals: {
    "redux-logger": "redux-logger", // This is described in optionalDependency
  },

A quick workaround is to chain optional packages with the nodeExternals() function like so:

externals: [
    nodeExternals(), {
    "redux-logger": "redux-logger",
  }],

I am not sure if this is fixable in any other way (maybe by checking the optional flag and calling the callback with the simplified name of the package...)
But this thing should at least be noted on the docs.

Thanks again for this package!

utils not found

Florians-MBP:collector falieson$ nps build.server
nps is executing `build.server` : npx webpack --config webpack/webpack.dev.server.js
Cannot find module './utils'

in my node_modules/webpack-node-externals/index.js
there is var utils = require('./utils');
but there is no utils in the folder

"webpack-node-externals": "1.7.1",

[Question] How to compile per file and keep structure?

Hello,

So i'm encounter of creating sub part as external application program that depending on other existing programs.
Example for that creating sub project to update some entries in database, so normally i use the existing driver to handle database call. but webpack is bundling all in one big bundle file!!

So how to bundle all files and keep structure of the application?

const path = require("path");
const fs = require("fs");
const CopyPlugin = require("copy-webpack-plugin");
const nodeExternals = require("webpack-node-externals");
const NodemonPlugin = require("nodemon-webpack-plugin");
const { CleanWebpackPlugin } = require("clean-webpack-plugin");

const ROOT_DIR = path.resolve(__dirname, "../../");
const APP_DIR = path.resolve(__dirname, "../../src");
const BUILD_DIR = path.resolve(__dirname, "../../build");

module.exports = {
  entry: `${APP_DIR}/server.ts`,
  output: {
    path: BUILD_DIR,
    filename: "server.js"
  },
  target: "node",
  externals: [nodeExternals()],
  resolve: {
    extensions: [".ts", ".js"],
    alias: {
      "@src": `${ROOT_DIR}/src/`,
      "@database": `${APP_DIR}/database/`
    }
  },
  plugins: [
    new CleanWebpackPlugin(),
    new CopyPlugin([{ from: `${APP_DIR}/public`, to: `${BUILD_DIR}/public` }]),
    new NodemonPlugin()
  ],
  module: {
    rules: [
      {
        test: /\.ts$/,
        use: ["ts-loader"],
        exclude: /node_modules/
      }
    ]
  }
};

It throws an error if there's no `node_modules`

If the process.cwd () doesn't contain a node_modules directory I get this error:

{ Error: ENOENT: no such file or directory, scandir 'node_modules'

Usually it wouldn't make sense to use this package in a project that doesn't even have a node_modules directory, but I'm using this package as part of a larger program for bundling files, and I think there's no need to throw an error in this case.

When I run the dev server, I got an error 'Uncaught ReferenceError: require is not defined' in external "url".

I have just started to study Webpack and trying to setup development environment based on Webpack4.

I put one script for executing dev server in package.json like bellow.

# package.json

	"scripts": {
		"dev": "webpack-dev-server --mode development",
	}

However, there was an error message like bellow when I execute 'npm run dev' on my terminal.

ERROR in ./node_modules/destroy/index.js
Module not found: Error: Can't resolve 'fs' in 'D:\webpack-setup\node_modules\destroy'
 @ ./node_modules/destroy/index.js 14:17-30

So, I installed 'webpack-node-externals' and put configuration in 'webpack.config.js' like bellow.

# install webpack-node-externals module

	# npm install --save-dev webpack-node-externals

# webpack.config.js

    const nodeExternals = require('webpack-node-externals');

	module.exports = {
		target: 'web',
		externals: [nodeExternals()]
		devServer: {
			host: 'localhost',
			port: 3000,
			open: true
		}
	};

When a browser was opened, there was an error on a browser like bellow.

Uncaught ReferenceError: require is not defined
    at eval (external_"url":1)
    at Object.url (main.js:289)
    at __webpack_require__ (main.js:20)
    at Object.eval (webpack:///(:3000/webpack)-dev-server/client?:6:11)
    at eval (webpack:///(:3000/webpack)-dev-server/client?:249:30)
    at Object../node_modules/webpack-dev-server/client/index.js?http://localhost:3000 (main.js:97)
    at __webpack_require__ (main.js:20)
    at eval (webpack:///multi_(:3000/webpack)-dev-server/client?:1:1)
    at Object.0 (main.js:190)
    at __webpack_require__ (main.js:20)

I'm not sure this error is related to 'webpack-node-externals' module or not,
but could I get some guide for solving this situation?

Can you please explain modulesFromFile config option?

I need to bundle explicit dependencies since I'm deploying to AWS Lambda.
How can I specifically include all of my project dependencies to be bundled and exclude all of my devDependencies and native node modules?

Documentation here is not clear at all. Should I parse the package.json myself and extract the dependency names as an array of strings?

Thanks.

Question | Why is not bundling node_modules good?

Hey,

I am trying to understand why not to bundle node_modules in our server bundle. I also read: http://jlongster.com/Backend-Apps-with-Webpack--Part-I, which from issue #1 seems to be the inspiration for this plugin, but there isn't a real explanation why not to bundle node_modules. In the above post, and in this plugin, you just state that it is not good, but why?

On the other hand, imho, bundling node_modules, makes deployment much smoother. For one, instead of having to copy the whole node_modules folder, you just copy the bundle, which is in our case is about 1Mb vs node_modules being 300+ Mbs. Secondly, when building into a dist folder (so we can have a single deployment point), trying to automatically copy node_modules into that folder, results in too many open files error.

Also (this is kind of a guess), correct me if I am wrong, but wouldn't having to load a "big" bundle file, would be like a one time thing while starting the server, and will only affect the initial load time of the server, which isn't that important any way?

transitive dependencies with npm link

I'm using npm-link to develop multiple related repos at once. So I use the whitelist to opt-in my related source modules (which are linked in node_modules). But I need node-externals to respect the dependencies in these other submodules.

Supporting multiple node_modules directories would probably resolve this. Happy to get feedback on other possible solutions.

Question

Thanks for putting this together. Just curious if there are any benefits over using this over what is recommended in http://jlongster.com/Backend-Apps-with-Webpack--Part-I

var webpack = require('webpack');
var path = require('path');
var fs = require('fs');

var nodeModules = {};
fs.readdirSync('node_modules')
  .filter(function(x) {
    return ['.bin'].indexOf(x) === -1;
  })
  .forEach(function(mod) {
    nodeModules[mod] = 'commonjs ' + mod;
  });

module.exports = {
  entry: './src/main.js',
  target: 'node',
  output: {
    path: path.join(__dirname, 'build'),
    filename: 'backend.js'
  },
  externals: nodeModules
}

best way to not externalize assets?

Sometimes you will be loading e.g. css files from an npm package, and those need to be converted so they cannot be external. This is what I came up with:

            nodeExternals({
                // load non-javascript files with extensions, presumably via loaders
                whitelist: [/\.(?!(js|json)$).{1,5}$/i],
            }),

That assumes that anything that has a max-5-char extension but is not js or json, will need to be parsed as an internal module.

Is this the best way? If so, perhaps put it in the Readme?

"Why is not bundling node_modules a good thing" info not correct?

It goes against the entire npm dependencies management. If you're using Lodash, and the consumer of your library also has the same Lodash dependency, npm makes sure that it will be added only once. But bundling Lodash in your library will actually make it included twice, since npm is no longer managing this dependency.

I'm not sure if this is correct. Webpack will only bundle it once if within the same version range. I think another reason to use this module is some modules might use binary. Some might use unconventional way to load modules. Some might use fs with relative paths.

Incorrectly classifies namespaced packages as external

My team has some packages that we bring in through npm namespaced under our organization (so, @shopify/package-name. We also have a local packages folder that contains packages we intend to release, but have not done so yet (so, packages/@shopify/local-package-name). Because of this line, which considers only the first part of a module name for the purpose of checking its presence in node_modules, all of our local packages are treated as externals and not bundled (we want them to be bundled for now since they are basically "app code", and because we don't want to mess with the NODE_PATH at runtime to include packages as a require-able directory).

Let me know if I can provide more details. Thanks for the great tool!

After upgrading from version 1.6.0 to 1.7.1 webpack fires an error stating it can't find ./utils

Thanks for making a great plugin.

Rolling back to version 1.6.0 and everything works fine. I'm using webpack 4.4.1.

My config is

const nodeExternals = require('webpack-node-externals')

module.exports = {
  target: 'node',
  externals: [nodeExternals()]
};

I see that the there is a new utils.js file in the project. I expect that could be the issue.
$ webpack --mode development
/Users/d0c/Projects/NODE/express_es2015_webpack/node_modules/webpack-cli/bin/webpack.js:242
throw err;
^

Error: Cannot find module './utils'
at Function.Module._resolveFilename (module.js:543:15)
at Function.Module._load (module.js:470:25)
at Module.require (module.js:593:17)
at require (/Users/d0c/Projects/NODE/express_es2015_webpack/node_modules/v8-compile-cache/v8-compile-cache.js:159:20)
at Object. (/Users/d0c/Projects/NODE/express_es2015_webpack/node_modules/webpack-node-externals/index.js:1:75)
at Module._compile (/Users/d0c/Projects/NODE/express_es2015_webpack/node_modules/v8-compile-cache/v8-compile-cache.js:178:30)
at Object.Module._extensions..js (module.js:660:10)
at Module.load (module.js:561:32)
at tryModuleLoad (module.js:501:12)
at Function.Module._load (module.js:493:3)
at Module.require (module.js:593:17)
at require (/Users/d0c/Projects/NODE/express_es2015_webpack/node_modules/v8-compile-cache/v8-compile-cache.js:159:20)
at Object. (/Users/d0c/Projects/NODE/express_es2015_webpack/webpack.config.js:1:85)
at Module._compile (/Users/d0c/Projects/NODE/express_es2015_webpack/node_modules/v8-compile-cache/v8-compile-cache.js:178:30)
at Object.Module._extensions..js (module.js:660:10)
at Module.load (module.js:561:32)
at tryModuleLoad (module.js:501:12)
at Function.Module._load (module.js:493:3)
at Module.require (module.js:593:17)
at require (/Users/d0c/Projects/NODE/express_es2015_webpack/node_modules/v8-compile-cache/v8-compile-cache.js:159:20)
at WEBPACK_OPTIONS (/Users/d0c/Projects/NODE/express_es2015_webpack/node_modules/webpack-cli/bin/convert-argv.js:133:13)
at requireConfig (/Users/d0c/Projects/NODE/express_es2015_webpack/node_modules/webpack-cli/bin/convert-argv.js:135:6)
at /Users/d0c/Projects/NODE/express_es2015_webpack/node_modules/webpack-cli/bin/convert-argv.js:142:17
at Array.forEach ()
at module.exports (/Users/d0c/Projects/NODE/express_es2015_webpack/node_modules/webpack-cli/bin/convert-argv.js:140:15)
at yargs.parse (/Users/d0c/Projects/NODE/express_es2015_webpack/node_modules/webpack-cli/bin/webpack.js:239:39)
at Object.parse (/Users/d0c/Projects/NODE/express_es2015_webpack/node_modules/yargs/yargs.js:543:18)
at /Users/d0c/Projects/NODE/express_es2015_webpack/node_modules/webpack-cli/bin/webpack.js:217:8
at Object. (/Users/d0c/Projects/NODE/express_es2015_webpack/node_modules/webpack-cli/bin/webpack.js:512:3)
at Module._compile (module.js:649:30)
error An unexpected error occurred: "Command failed.
Exit code: 1
Command: sh
Arguments: -c webpack --mode development
Directory: /Users/d0c/Projects/NODE/express_es2015_webpack
Output:
".
info If you think this is a bug, please open a bug report with the information provided in "/Users/d0c/Projects/NODE/express_es2015_webpack/yarn-error.log".
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

Is this needed for Webpack @ 2?

I am using the beta version of Webpack V2. Just wondering if this is still needed with that or did they build something in to do this? My gut says its still needed, because I haven't figured out a way around it yet. :)

Deleted

Deleted content -- due to misunderstanding

weback hot expressjs critical dependency

Hi, getting the Critical Dependency error when using express with hot module.

WARNING in ./packages/servers/express/node_modules/colors/lib/colors.js
127:29-43 Critical dependency: the request of a dependency is an expression

WARNING in ./packages/servers/express/node_modules/express/lib/view.js
79:29-41 Critical dependency: the request of a dependency is an expression

webpack.config.server.js

const webpack = require('webpack');
const path = require('path');
const nodeExternals = require('webpack-node-externals');
const StartServerPlugin = require('start-server-webpack-plugin');

module.exports = {
  entry: [
    'babel-polyfill',
    'webpack/hot/poll?1000',
    path.join(__dirname, '../packages/servers/express/server/index.js')
  ],
  watch: true,
  target: 'node',
  externals: [
    nodeExternals({
      whitelist: ['webpack/hot/poll?1000']
    })
  ],
  resolve: {
    alias: {
      handlebars: 'handlebars/dist/handlebars.js'
    }
  },
  module: {
    rules: [
      {
        test: /\.js?$/,
        use: 'babel-loader',
        exclude: /node_modules/
      }
    ]
  },
  plugins: [
    new StartServerPlugin('server.js'),
    new webpack.NamedModulesPlugin(),
    new webpack.HotModuleReplacementPlugin(),
    new webpack.NoEmitOnErrorsPlugin(),
    new webpack.DefinePlugin({
      'process.env': { BUILD_TARGET: JSON.stringify('server') }
    })
  ],
  output: {
    path: path.join(__dirname, '../packages/servers/express/.build'),
    filename: 'server.js'
  }
};

Thanks

does it work in 2.1.0 ?

I'm trying to use it to ignore standard node modules:

import nodeExternals from 'webpack-node-externals'

const config = {
  ...
  externals: [nodeExternals()],
  target: 'node',
}

but getting this:

Hash: 622e9456d2604a290a58
Version: webpack 2.1.0-beta.27
Time: 209ms
    Asset     Size  Chunks             Chunk Names
bundle.js  4.34 kB       0  [emitted]  main
chunk    {0} bundle.js (main) 366 bytes [entry] [rendered]
    [0] external "path" 42 bytes {0} [not cacheable]
    [1] external "util" 42 bytes {0} [not cacheable]
    [2] ./src/test.js 282 bytes {0} [built]

(test.js imports path and util)
any chance this might be related to context configuration or any other value?

Need help with more details on how this webpack-node-externals works

According to Readme, webpack-node-externals exclude any node_modules on server side.

I have a React SSR project and in webpack config for server side, I added this plugin in externals fields. On server side, I still need 'react' and 'react-dom' and react-apollo. It works fine but it suddenly makes me confused if all node_modules are excluded , how come react, react dom still work on server side?

Then I added a third party react component library and suddenly all exported stuff from it on server is undefined.

Please help me with my understanding

Thanks!

Just wanted to say thanks for an amazing plugin.

not working with import statements

I can't seem to get this working with import statements. The modules are still bundled if I use import in my code. However, if I use require(), it works fine.

Am I missing something?

Can't resolve 'webpack/hot'

Hello,
I'm trying to use nodeExternals for a React app that has SSR and trying to get HMR working. In my server config I've added nodeExternals as such:

nodeExternals({whitelist: ['webpack/hot/dev-server']})

Unfortunately I'm subsequently seeing this error:

ERROR in (webpack)-dev-server/client?http://localhost:9000
    Module not found: Error: Can't resolve 'webpack/hot' in '/Users/mprzybylski/Desktop/ace-test/node_modules/webpack-dev-server/client'
     @ (webpack)-dev-server/client?http://localhost:9000 95:17-67
     @ multi (webpack)-dev-server/client?http://localhost:9000 (webpack)/hot/dev-server.js ./src/server.js

Have I set something up incorrectly? Any advice would be greatly appreciated.

"SyntaxError: Unexpected token ." in webpack script

Hello,
I have the following configuration:

node 10.16.0
electron 3.0.2
webpack 4.20.2
webpack-node-externals 1.7.2

When adding the line externals: [nodeExternals()], in my webpack script I get the error:

vm.js:74 Uncaught SyntaxError: Unexpected token .
    at new Script (vm.js:74)
    at createScript (vm.js:246)
    at Object.runInThisContext (vm.js:298)
    at Module._compile (internal/modules/cjs/loader.js:678)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:722)
    at Module.load (internal/modules/cjs/loader.js:620)
    at tryModuleLoad (internal/modules/cjs/loader.js:559)
    at Function.Module._load (internal/modules/cjs/loader.js:551)
    at Module.require (internal/modules/cjs/loader.js:658)
    at require (internal/modules/cjs/helpers.js:20)

I tracked down the code segment to:

   try {
      super(code,
            filename,
            lineOffset,
            columnOffset,
            cachedData,
            produceCachedData,
            parsingContext);
    } catch (e) {
      throw e; /* node-do-not-add-exception-line */
    }

So I assume the loaded code is having a syntax error. I looked into the code fragment and for an unknown reason it pulls in a css file and wraps it in js code for whatever reason:

(function (exports, require, module, __filename, __dirname, process, global, Buffer) { 
    return function (exports, require, module, __filename, __dirname) { 
        .srd-diagram{position:relative;flex-grow:1;display:flex;cursor:move;overflow:hidden}
        .srd-diagram__selector{position:absolute;background-color:rgba(0,192,255,0.2);
        border:solid 2px #00c0ff}
        ...
}

I thought that the issue might be in my webpack.config.js but it runs without the external module command.

var path = require('path');
const ExtractTextPlugin = require("extract-text-webpack-plugin");
var nodeExternals = require('webpack-node-externals');

module.exports = {
  watch: true,
  // target: "electron-renderer",
  target:"node",
  entry: "./app/src/renderer_process.jsx",
  output: {
    path: path.resolve(path.join(__dirname,"app","build")),
    publicPath: "build/",
    filename: "bundle.js"
  },
  // externals: ['grpc'],
  externals: [nodeExternals()],
  node: {
    __dirname:  true,
    __filename: false
  },
  module: {
    rules: [{
        test: /\.jsx?$/,
        // loader: "babel-loader",
        exclude: /node_modules/,
        use: {
          loader: "babel-loader",
          options: {
            presets: ["babel-preset-env"]
          }
        }
      },
      {
        test: /\.css$/,
        use: [{
          loader: "style-loader/url"
        }, {
          loader: "file-loader"
        }]
      },
      {
        test: /\.scss$/,
        use: [{
            loader: "style-loader" // creates style nodes from JS strings
          },
          {
            loader: "css-loader" // translates CSS into CommonJS
          },
          {
            loader: "sass-loader" // compiles Sass to CSS
          }
        ]
      },
      {
        test: /\.(png|jpg|gif|svg)$/,
        loader: "file-loader",
        query: {
          name: "[name].[ext]?[hash]"
        }
      }
    ]
  },

  plugins: [
    new ExtractTextPlugin({
      filename: "bundle.css",
      disable: false,
      allChunks: true
    })
  ],

  resolve: {
    extensions: [".js", ".jsx", ".json"]
  }
};

Any ideas?

Multiple `modulesDir`s

In my project i has common package.json with common modules and child modules with their specific libs. So for modulesDir option i need to define both node_modules folders for proper bundling.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.