Comments (22)
I would like to reproduce your build. I tried alamid/test/exampleApp
but there seem to be no implementation in some services. (which only gives a weird error msg, see below)
The bundling is a complex process, but your case should be below a second.
- Don't use slow/sync loader/preprocessors. You can check if the compilation is cpu bound.
- Check that you don't use production-only features in development, they are slow: i. e. minimizing.
- Use the webpack cache. (!!! This gives the most speed)
I'm trying to optimize preformance. So there are some features planed:
- Multi process compilation, because compilation is mostly cpu bound.
- Store the cache not only in memory but also in filesystem to reuse it.
- preformance profiling for loader etc.
- If you have more hint, your welcome
You may send me your flame graph, that would be cool :)
The most time consuming part of webpack compiling are:
- path resolution, because there are many paths which need to be tested.
- javascript parsing esprima
- loaders, i. e. using coffeescript/css/less takes twice the time
error for alamid/test/exampleApp
, because of missing exports from services.
C:\Users\Sokrates\Eigene Repos\node_modules\alamid\lib\core\helpers\instantiateClasses.js:12
classes[identifier] = new Class();
^
TypeError: object is not a function
at C:\Users\Sokrates\Eigene Repos\node_modules\alamid\lib\core\helpers\instantiateClasses.js:12:31
at Function._.each._.forEach (C:\Users\Sokrates\Eigene Repos\node_modules\alamid\node_modules\underscore\underscore.js:84:24)
at instantiateClasses (C:\Users\Sokrates\Eigene Repos\node_modules\alamid\lib\core\helpers\instantiateClasses.js:11:7)
at populateServiceRegistry (C:\Users\Sokrates\Eigene Repos\node_modules\alamid\lib\server\bootstrap.server.js:31:32)
at Object.bootstrap [as startServer] (C:\Users\Sokrates\Eigene Repos\node_modules\alamid\lib\server\bootstrap.server.js:87:29)
at Object.<anonymous> (C:\Users\Sokrates\Eigene Repos\node_modules\alamid\test\exampleApp\app\init.server.js:6:8)
at Module._compile (module.js:449:26)
at Object.Module._extensions..js (module.js:467:10)
at Module.load (module.js:356:32)
at Function.Module._load (module.js:312:12)
from webpack.
Thanks for investigating. I will try to provide you with a flame graph so we can sort it out if we're responsible for this mess 😉. Also I will try the caching feature to improve loader performance.
You could take a look at fs.realpath. I think it's better than fs.exists
because you can provide a caching object which reduces the number of fs.stat
-calls. This caching object should be empty for every new bundler-call of course.
from webpack.
Ah btw: I'm not sure what the current development state of alamid/core is but in order to run the bundle tests you can just try node node_modules/mocha/bin/_mocha test/core/bundle/createBundle.test.js
in the root-folder. This worked for me with the current revision.
from webpack.
Why do you always use this.cacheable && this.cacheable();
with condition?
from webpack.
I thought that loaders may be used in another module system which do not support cacheable
as I specified it here as optional. If you only use webpack, it can be omitted ;)
from webpack.
your nodeclass loader is really slow. removing it gives 30% speedup :-|
It require
the file, cross-compiles the file and vm.runInNewContext
the result. This are 3 compilations... Why is this so complex? What makes nodeclass so special?
from webpack.
In my recent profiling tests nodeclass
was inconspicuous but I'll take a look on it :). Currently it seems to increase packaging time between 200-300ms.
nodeclass
is stuck between two development stages. I've built it a year ago and started to refactor it for 6 months. Unfortunately I haven't found the time to refactor it properly so currently there are different concepts mixed together.
from webpack.
All modules are cachable so using watch mode should be very fast at the second compilation. You should integrate some kind to dev-server into alamid, which recompiles on change (i. e. use webpack-dev-middleware).
You compilation is cpu bound so there are no big issues which blocking stuff. Multi process compilation should give a big speedup (next version).
from webpack.
Is caching also applied when using webpack without watch mode (with the node process still running)?
from webpack.
Not by default. But you can pass a options.cache
. It should be a new instance of webpack/lib/Cache
. See here.
from webpack.
There is info in the stats
if a module was laoded from cache (fromCache
) or stored to cache (toCache
). If there is no info but a cache was used the module was not cachable.
from webpack.
Made performance improvements in 0.6.x and there is [email protected]. Also multi process building is availible, but only recommended for bigger projects. (there is a process fork overhead of ~100-200ms)
Also check --profile
or options.profile=true
for finding proformance problems in loaders. Try the print the output with:
var formatOutput = require("webpack/lib/formatOutput");
console.log(formatOutput(stats, {
colors: true,
context: options.context
});
or use the command line
from webpack.
Nice! Looks good, I'll try it 😄
from webpack.
It seems like the raw-loader is somehow influencing the resolving time. Don't know what's going on but if a module is yellow or red the raw-loader is involved in 80%.
/home/roomie/node/alamid/~/raw-loader!./app/pages/MainPage.html [1220ms: 1167ms resolving, 0ms waiting, 52ms build, 1ms children]
(webpack)/~/bundle-loader/lazy.js!/home/roomie/node/alamid/~/raw-loader!./app/pages/home/HomePage.html [1496ms: 1327ms resolving, 0ms waiting, 50ms build, 119ms children]
/home/roomie/node/alamid/~/raw-loader!./app/pages/blog/BlogPage.html [979ms: 925ms resolving, 0ms waiting, 53ms build, 1ms children]
I'm using a postprocessor (which could influence the resolving time), but when I'm profiling it it doesn't take more than 1ms.
from webpack.
Webpack has to check many module folders to find it:
./app/pages/node_modules
./app/pages/web_modules
./app/node_modules
- ...
...alamid/node_modules
(see options.resolve.modulesDirectorys
)
Each time it has to check raw-webpack-web-loader
, raw-webpack-loader
, raw-web-loader
, raw-loader
and raw
. (see options.resolve.loaderPostfixes
)
Than it has to check and read the package.json
.
Than it has to check index.webpack-loader.js
, index.webpack-web-loader.js
, index.web-loader.js
, index
and index.js
. (see options.resolve.loaderExtensions
)
That's a bit slow... (TODO optimize it)
You can try to add ...node/alamid/node_modules
to options.paths
, maybe that makes it a bit faster.
from webpack.
You can also try to reduce to lookup dirs/extensions/postfixes to that you want to use.
That are the defaults (see README)
options.resolve.extensions = ["", ".webpack.js", ".web.js", ".js"];
options.resolve.postfixes = ["", "-webpack", "-web"];
options.resolve.loaderExtensions = [".webpack-web-loader.js", ".webpack-loader.js", ".web-loader.js", ".loader.js", "", ".js"];
options.resolve.loaderPostfixes = ["-webpack-web-loader", "-webpack-loader", "-web-loader", "-loader", ""];
options.resolve.modulesDirectorys = ["web_modules", "jam", "node_modules"];
from webpack.
Performance checklist: (ordered by priority)
- use caching
- use watch mode, or cache object
- flag loaders cachable
- optimize resolving
- minimize resolve checks in your settings
- use worker processes
- file an issue with new ideas how to optimize performance
- cry that your hyper-complex project takes multiple seconds to compile
I'll close that issue for now...
from webpack.
@sokra I'm trying to get the cache working without using watch mode. All of my loaders are cacheable.
Passing in an empty object in options.cache
does not seems to actually cache. The stats.toCache
and stats.fromCache
flags are always undefined
.
Is the CachePlugin supposed to be used in conjunction with this?
from webpack.
Yes, I think so. A lot has changed since then.
Reading from the source code you could try this:
// webpack.config.js
var CachePlugin = require("webpack/lib/CachePlugin")
var myCache = {};
module.exports = {
...
plugins: [
new CachePlugin(myCache);
]
};
However, it is still important that the node.js process keeps running. You need to keep a reference on the myCache
object.
from webpack.
It looks like all I need to provide in the config is an empty object in options.cache
. The WebpackOptionsApply module seems to create the cache plugin for us:
if(options.cache === undefined ? options.watch : options.cache) {
var CachePlugin = require("./CachePlugin");
compiler.apply(new CachePlugin(typeof options.cache === "object" ? options.cache : null));
}
I tested both adding the CachePlugin instance to the list of plugins as well as just passing in an empty object (and both) but the stats.toCache
and stats.fromCache
are always undefined
. Subsequent builds take the same duration even though the cache
object gets populated.
On a side note, it looks like the Webpack module exports the CachePlugin.
from webpack.
It turns out I was using gulp-webpack which creates a new compiler instance every time thus ignoring whatever was put in the cache. Solved by simply using webpack and using the same compiler every run.
Thanks for the help!
from webpack.
A new compiler instance every time should be just fine according to the docs as long as you store the cache object somewhere where it remains alive between compiler instantiations.
from webpack.
Related Issues (20)
- `snapshot.unmanagedPaths` is ignored
- Module Federation - Some chunks are not loaded when navigating from the Remote module to the Host module
- failed to import ffmpeg-core.js HOT 1
- Webpack 5 doesnt generate the chunks of imports HOT 1
- Error: Cannot find module 'webpack-cli/package.json' HOT 2
- Unable to bust cache of module federation remotes....
- Re-exported Ant Design components don't have same style with the original components
- failed to import ffmpeg-core.js HOT 3
- DllPlugin: When processing multiple chunks, does the `path` option must have a string template?
- Webpack BannerPlugin with `raw` option should put `use strict` on the very first line HOT 10
- After webpack 5 migration not able to use local packages in a mono repo
- 95% emitting emit phase takes much longer in Node >=16
- Plugin apply not working for multi compiler HOT 2
- packages/tailwindcss/src/compile.ts HOT 1
- Current error handling mainly involves calling the `onInvalidCandidate` callback function to handle invalid candidates, and printing corresponding error messages to the console.
- [5.91.0] "Fixed fs types" breaks `outputFileSystem` compatibility with `memfs` HOT 6
- Using `[name]` within `output.library`, the correct value isn't utilized in the source map namespace HOT 1
- Not found node_modules/@angular/common/bundles/common-http.umd.js
- webpack5.90.0 Modifying files in the node_modules directory (local component library) cannot trigger hot updates HOT 1
- Delete this. HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from webpack.