magento / baler Goto Github PK
View Code? Open in Web Editor NEWAMD module bundler and preloader for Magento 2 stores.
License: Open Software License 3.0
AMD module bundler and preloader for Magento 2 stores.
License: Open Software License 3.0
These Mirasvit modules are really throwing some curve balls.
Error: Found > 1 "module.xml" for the module at path "vendor/mirasvit/module-search"
at findRealModuleRoot (/Users/scottfriend/Sites/baler/src/magentoFS.ts:247:15)
at processTicksAndRejections (internal/process/task_queues.js:89:5)
at getModuleConfig (/Users/scottfriend/Sites/baler/src/magentoFS.ts:204:28)
at async Promise.all (index 151)
at getComposerComponents (/Users/scottfriend/Sites/baler/src/magentoFS.ts:332:18)
at async Promise.all (index 0)
at Object.getComponents (/Users/scottfriend/Sites/baler/src/magentoFS.ts:62:37)
at async Promise.all (index 1)
at Object.collectStoreData (/Users/scottfriend/Sites/baler/src/collectStoreData.ts:15:58)
at Object.run (/Users/scottfriend/Sites/baler/src/cli.ts:20:19)
Here's the abbreviated directory structure of this module:
vendor/mirasvit/module-search
I'm really not sure why it makes sense to have two Mage modules inside one composer module =/
Hello,
I followed all the steps in https://github.com/magento/baler/blob/master/docs/ALPHA.md
And when I ran the command 'baler' everything was looking fine. For each Magento theme I got the command line message 'Created core bundle file' and 'Minified core bundle and RequireJS', however I'm not seeing any bundle in the project. Also checking the store url I get all JS files separate so no actual bundle was generated.
Am I missing something?
Kind regards,
EDIT: Found the bundle under balerbundles/core-bundle.js
Question: Can this speed up Magento Admin too?
Hello,
can someone suggest where i can find initializeCLI
this file and dist directory inluded in barel
root@vmi223508:~/baler# tail -1 bin/baler
require('../dist/cli/initializeCLI').initializeCLI(process.argv, process.cwd());
but dist directory does not exist
root@vmi223508:~/baler# ls
bin docs jest.config.js node_modules package.json package-lock.json prettier.config.js prettier-globs README.md src tsconfig.json
due to this barel produce error below
root@vmi223508:/baler# node -v/baler# baler -v
v12.13.1
root@vmi223508:
Error: Cannot find module '../dist/cli/initializeCLI'
Require stack:
I also try find similar file and directory in magento installed directory, but also without success
Can someone suggest, where I can find dist directory with related files
I'm aware that this project is still in its infancy but I'm pleased to see it developing so well!
When running baler in the root of my Magento 2 store, I get an immediate error
Error: ENOENT: no such file or directory, open
complaining that '/etc/module.xml' does not exist in one of our third-party modules in the vendor directory. This is true because this vendor has decided to place their 2 Magento 2 modules in the same composer package. Therefore the path to each should be 'vendor/vendorname/modulename/src/module1/etc/module.xml' etc.
If this is a complicated issue to solve then some documented way to disable some modules from being bundled would also be great.
I got following Error when i visit my homepage
Uncaught TypeError: Cannot read property 'setConf' of undefined customer-data.js:33
at customer-data.js:33
at Object.execCb (require.js:1650)
at Module.check (require.js:866)
at Module.<anonymous> (require.js:1113)
at require.js:132
at require.js:1156
at each (require.js:57)
at Module.emit (require.js:1155)
at Module.check (require.js:917)
at Module.enable (require.js:1143)
findMagentoRoot
check requirements.Currently Baler checks for these required directories:
However, index.php
is not necessary, while vendor
probably is necessary (for the graph building later on).
The baler build
should preferably only check for pub/static/frontend
because that's the only place it needs to get data from, and it's enough to determine you are in the right directory.
For the baler graph
method (when released) you could do a wider check, including the other directories.
1 - The error message could be more descriptive (currently says Could not find Magento store root from [X]
.
2 - The requirements could be reduced
A lot of us devs out there use docker and run npm
commands locally, not in the containers. So we usually don't have all these directories locally mounted/synced to the containers.
Especially lib/
and index.php
are very rare to have synced.
Unsure whether this is or isn't a Baler issue, or something modulemakers could be doing wrong.
When running 'baler/bin/baler' from the root i get the following message:
Error: ENOENT: no such file or directory, open [domain]/vendor/mirasvit/module-helpdesk/etc/module.xml
My assumption here is that baler is spitting through modules to find dependencies and possible JS files to bundle?
This is actually correct though, the location it should be looking for is vendor/mirasvit/module-helpdesk/src/Helpdesk/etc/module.xml
. I believe the module maps these locations through composer.json:
It looks like there's an assumption about where baler should find vendor files, based on the generated files? This assumption then doesn't take into account that modulemakers might map their modules in a different way.
I could be totally off-base here, but hope this helps make baler the tool we'll all be using in a month (or two ;))
Hi!
First of all, thank you working on Baler. I'm not using it in production yet but preliminary results are looking promising!
I've had it working with only minor hick-ups on 2.3.3 but this morning I tried to build 2.3.4 and got a crash in the frontend. It seems like Baler assumes that text!
-dependencies are always HTML files and so things like 'text!js-translation.json'
(introduced here) doesn't work.
I did the simplest fix I could think of just to fix my crash:
diff --git a/src/createBundleFromDeps.ts b/src/createBundleFromDeps.ts
index 897a047..3fab6f4 100644
--- a/src/createBundleFromDeps.ts
+++ b/src/createBundleFromDeps.ts
@@ -58,12 +58,13 @@ async function getFinalModuleSource(
const path = join(baseDir, resolver(dep).modulePath);
const source = await readFile(path, 'utf8');
const isHTML = extname(path) === '.html';
+ const isJSON = extname(path) === '.json';
const shims = getShimsForModule(dep, requireConfig);
const hasDefine = isAMDWithDefine(source);
const isNamed = isNamedAMD(source);
const hasInvalidShim = hasDefine && !!shims;
- if (isHTML) {
+ if (isHTML || isJSON) {
return { dep, file: wrapTextModule(dep, source), hasInvalidShim };
}
But I suppose a better long-term fix would be to look at the text!
-prefix instead of the file extension. What do you think?
Cheers!
An example of an adhoc script can be found, but we should probably just use something off-the-shelf (suggestions in the linked comment)
Hey! First of all I need to say that I'm really impressed with the performance of this solution an ease of use in relation to other tools which take advantage of headless browsers etc., great work!
Unfortunately, when I gave baler for a spin it seems like we hit the same obstacle that I did and documented under https://github.com/magesuite/magepack#modules-with-mixins-defined-cannot-be-included-in-a-bundle
The issue comes from mixins.js file and its processNames
method which is (as far as I understand) responsible for prepending given modules' names with mixins!
prefix in case certain module has any mixins defined for it so that they can be applied when it is loaded. The problem is that in order to get a path for said module, getPath
is called, which then calls require.toUrl
. Multiple calls later we land inside require's nameToUrl
method:
nameToUrl: function (moduleName, ext, skipExt) {
var paths, syms, i, parentModule, url,
parentPath, bundleId,
pkgMain = getOwn(config.pkgs, moduleName);
if (pkgMain) {
moduleName = pkgMain;
}
bundleId = getOwn(bundlesMap, moduleName); // OH NO
if (bundleId) { // NO NO NO PLEASE NO
return context.nameToUrl(bundleId, ext, skipExt);
}
As you may have figured out, for each of our modules that were included in core bundle we are getting balerbundles/core-bundle
instead of the actual name. Because of the fact that stock Magento makes use of mixins, even for providing jquery security patches I would say this issue is critical and should be addressed ASAP before anyone uses this solution in production.
There are 2 possible ways this can be solved:
mixins.js
bundling-aware by reimplementing whole toUrl
to nameToUrl
chain so that it doesn't have if (bundleId) {
check in it.That's all that I can see right now, I'll try to hack around a bit to see what the simplest route would be.
It seems that baler needs a PHP executable. This is not documented. Of course if you work with Magento you usually also have PHP installed, but I was building a Docker image for baler and didn't realize that PHP was also directly required for executing baler.
It would be great, if the documentation included that information.
Also why does baler need a PHP executable...?
When I try to run baler, I get the following error:
internal/modules/cjs/loader.js:638
throw err;
^Error: Cannot find module 'source-map-support/register'
at Function.Module._resolveFilename (internal/modules/cjs/loader.js:636:15)
at Function.Module._load (internal/modules/cjs/loader.js:562:25)
at Module.require (internal/modules/cjs/loader.js:692:17)
at require (internal/modules/cjs/helpers.js:25:18)
at Object. (/home/apache/.local/bin/baler:35:1)
at Module._compile (internal/modules/cjs/loader.js:778:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:789:10)
at Module.load (internal/modules/cjs/loader.js:653:32)
at tryModuleLoad (internal/modules/cjs/loader.js:593:12)
at Function.Module._load (internal/modules/cjs/loader.js:585:3)
I tried to use
npm install
in the root folder of baler.
And I've got an error:
npm ERR! Maximum call stack size exceeded
baler
has 2 pieces that are necessary for bundling to work within Magento.
baler
- an npm
package used to package bundles and generate configurationMagento_Baler
- a Magento module that injects necessary <link rel="preload" />
tags into the head
of the documentMagento_Baler
baler
When baler
is run, is spits out 2 different types of artifacts
core-bundle.js
baler-preload-map.json
(see #3)Magento_Baler
is an enabled module in the storerequirejs-baler-config.js
On each request:
requirejs-config.js
in the head
of the document with requirejs-baler-config.js
head
of the document, add a preload
tag that points to balerbundles/core-bundle.js
in the root of the current theme's current localeOn each request
.phtml
files being used to render the current requestbaler-preload-map.json
, which is a sibling of core-bundle.js
in the root of the locale.preloads
baler-preload-map.json
:
.phtml
file at that path was used to renderpreloads
preloads
listpreloads
, add a preload
tag to the head
of the documentThe following features should be disabled when Magento_Baler
is enabled:
.phtml
files loaded keeps the order these were loaded on the page. When injecting preload
tags in the head
, inject them in the same order that the .phtml
files were rendered (this effectively prioritizes content higher in the document)I have been struggling with getting baler working in my magento 2 store where i did run de build command but stays stuck at creating core bundle file. It has been running for 24 hours now. I have also followed multiple tutorials on how to install it so im sure where it goes wrong.
Screenshot: https://prnt.sc/rzdeo8
Hey @DrewML
We're using Baler for projects, but since today we're unable to deploy because Baler gives an error. Please see error code below. We don't see any new commits in Baler and the commits that worked before in our projects are failing now as well (no code changes). We've been able to reproduce this in our development environment as well. Changing "default" to another name seems to resolve the issue.
baler stderr: /chroot/home/nextmp/.config/yarn/global/node_modules/@magento/baler/dist/createBundleFromDeps.js:14
Object.defineProperty(o, "default", { enumerable: true, value: v });
^
TypeError: Cannot redefine property: default
at Function.defineProperty (<anonymous>)
at __setModuleDefault (/chroot/home/nextmp/.config/yarn/global/node_modules/@magento/baler/dist/createBundleFromDeps.js:14:12)
at __importStar (/chroot/home/nextmp/.config/yarn/global/node_modules/@magento/baler/dist/createBundleFromDeps.js:22:5)
at Object.<anonymous> (/chroot/home/nextmp/.config/yarn/global/node_modules/@magento/baler/src/createBundleFromDeps.ts:18:1)
at Module._compile (internal/modules/cjs/loader.js:778:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:789:10)
at Module.load (internal/modules/cjs/loader.js:653:32)
at tryModuleLoad (internal/modules/cjs/loader.js:593:12)
at Function.Module._load (internal/modules/cjs/loader.js:585:3)
at Module.require (internal/modules/cjs/loader.js:692:17)
from /var/lib/gems/2.5.0/gems/sshkit-1.21.0/lib/sshkit/runners/parallel.rb:11:in `block (2 levels) in execute'
That is a list of loaded JS files in waterfall order.
Pay attention to the order of files core-bundle.js and jquery.cookie.js
core-bundle.js includes jquery-storageapi.js at the same time, storageapi is depended from
'jquery/jquery-storageapi': {'deps': ['jquery/jquery.cookie']}
As you see from Image jquery.cookie.js is loaded later bundle. In result, I have some Js error because of undefined Jquery methods.
Looks like all files included in the bundle should be checked for dependencies, and I afraid they should be sorted in the bundle in the correct order.
I assume, that the problem has to do with insufficient memory on my vserver but i just wanted to ask if there is something i can fix to get baler running.
Running on Magento 2.3.4 and getting the following ERROR when trying to run baler.
[risingbytes@np500 public_html]$ baler baler ✔ Collected theme/module data 91ms /home/risingbytes/.nvm/versions/node/v13.7.0/bin/node[17854]: ../src/node_platform.cc:63:std::unique_ptr<long unsigned int> node::WorkerThreadsTaskRunner::DelayedTaskScheduler::Start(): Assertion
(0) == (uv_thread_create(t.get(), start_thread, this))' failed.
node[17847]: pthread_create: Resource temporarily unavailable
1: 0xa9d570 node::Abort() [/home/risingbytes/.nvm/versions/node/v13.7.0/bin/node]
2: 0xa9d5f7 [/home/risingbytes/.nvm/versions/node/v13.7.0/bin/node]
3: 0xb082aa node::WorkerThreadsTaskRunner::WorkerThreadsTaskRunner(int) [/home/risingbytes/.nvm/versions/node/v13.7.0/bin/node]
/home/risingbytes/.nvm/versions/node/v13.7.0/bin/node[17865]: ../src/node_platform.cc:63:std::unique_ptr node::WorkerThreadsTaskRunner::DelayedTaskScheduler::Start(): Assertion `(0) == (uv_thread_create(t.get(), start_thread, this))' failed.
4: 0xb0836b node::NodePlatform::NodePlatform(int, node::tracing::TracingController*) [/home/risingbytes/.nvm/versions/node/v13.7.0/bin/node]
5: 0xa6e1a1 node::InitializeOncePerProcess(int, char**) [/home/risingbytes/.nvm/versions/node/v13.7.0/bin/node]
node[17853]: pthread_create: Resource temporarily unavailable
1: 0xa9d570 node::Abort() [/home/risingbytes/.nvm/versions/node/v13.7.0/bin/node]
6: 0xa6e6a3 node::Start(int, char**) [/home/risingbytes/.nvm/versions/node/v13.7.0/bin/node]
7: 0x7f3c7abe0545 __libc_start_main [/lib64/libc.so.6]
2: 0xa9d5f7 [/home/risingbytes/.nvm/versions/node/v13.7.0/bin/node]
8: 0xa0afc5 [/home/risingbytes/.nvm/versions/node/v13.7.0/bin/node]
3: 0xb082aa node::WorkerThreadsTaskRunner::WorkerThreadsTaskRunner(int) [/home/risingbytes/.nvm/versions/node/v13.7.0/bin/node]
4: 0xb0836b node::NodePlatform::NodePlatform(int, node::tracing::TracingController*) [/home/risingbytes/.nvm/versions/node/v13.7.0/bin/node]
5: 0xa6e1a1 node::InitializeOncePerProcess(int, char**) [/home/risingbytes/.nvm/versions/node/v13.7.0/bin/node]
6: 0xa6e6a3 node::Start(int, char**) [/home/risingbytes/.nvm/versions/node/v13.7.0/bin/node]
7: 0x7f5813e93545 __libc_start_main [/lib64/libc.so.6]
8: 0xa0afc5 [/home/risingbytes/.nvm/versions/node/v13.7.0/bin/node]
Error: spawn ENOMEM
at ChildProcess.spawn (internal/child_process.js:405:11)
at spawn (child_process.js:548:9)
at Object.fork (child_process.js:116:10)
at ChildProcessWorker.initialize (/home/risingbytes/baler/node_modules/jest-worker/build/workers/ChildProcessWorker.js:137:44)
at new ChildProcessWorker (/home/risingbytes/baler/node_modules/jest-worker/build/workers/ChildProcessWorker.js:127:10)
at WorkerPool.createWorker (/home/risingbytes/baler/node_modules/jest-worker/build/WorkerPool.js:44:12)
at new BaseWorkerPool (/home/risingbytes/baler/node_modules/jest-worker/build/base/BaseWorkerPool.js:82:27)
at new WorkerPool (/home/risingbytes/baler/node_modules/jest-worker/build/WorkerPool.js:30:1)
at new JestWorker (/home/risingbytes/baler/node_modules/jest-worker/build/index.js:131:26)
at Object.createMinifier (/home/risingbytes/baler/src/createMinifier.ts:9:21)
`
No work to be done on this project, just keeping this here for tracking/visibility
For the Graph Preloading
functionality, it's necessary for baler
to spit out a mapping of JavaScript dependencies needed per .phtml
file.
baler-preload-map.json
, and should be written to the root of a locale dir, as a sibling of requirejs-baler-config.js
Magento_Baler
to use for validation.phtml
file{
"vendor/magento/module-example/view/frontend/templates/some-template.phtml": ["path/to/file/1.js", "path/to/file/2.js"],
"vendor/thirdparty/module-bar/view/frontend/templates/another-template.phtml": ["another/file.js", "yet/another/file/2.js"]
}
When running baler build
with our themes it seems to not complete the task and create any files and gives no output as to what could be going wrong, but seems to work fine for the Magento/luma theme:
Optimization is done, but stats have not been implemented in the CLI yet
- Collect theme/module data
✔ Collected theme/module data 250ms
- (Vendor/parenttheme) Create core bundle file
- (Vendor/childtheme) Create core bundle file
- (Magento/luma) Create core bundle file
✔ (Magento/luma) Created core bundle file 773ms
- (Magento/luma) Minify core bundle and RequireJS config
✔ (Magento/luma) Minified core bundle and RequireJS 10.5s
This is using the current master branch and v10.17.0 of node.
trace output:
(450.8348079994321): Resolving dependency "jquery/jquery.mobile.custom" from "unknown source"
(450.95511799678206): Resolving dependency "mage/common" from "unknown source"
(451.0416179969907): Resolving dependency "mage/dataPost" from "unknown source"
[...]
(452.7179719991982): Resolving dependency "Magento_Catalog/js/more-less" from "unknown source"
(452.83620499819517): Preparing to analyze "jquery/jquery.mobile.custom" for dependencies
(453.26991099864244): Warning for missing dependency "jquery/jquery.mobile.custom", which was required by "<entry point>"
(453.3077109977603): Preparing to analyze "mage/common" for dependencies
(453.3644319996238): Warning for missing dependency "mage/common", which was required by "<entry point>"
(453.37505899742246): Preparing to analyze "mage/dataPost" for dependencies
(453.4178909994662): Warning for missing dependency "mage/dataPost", which was required by "<entry point>"
[...]
(468.5774130001664): Resolving dependency "Magento_Catalog/js/more-less" from "unknown source"
baler then gets stuck in an endless loop.
New line is missing after sourceMappingURL
hint causing next merged file be on the same line as the comment:
// some code
gento_Theme/js/theme"]})}(require);
//# sourceMappingURL=requirejs-bundle-config.js.map;try { <==== this is a problem
if (!window.localStorage || !window.sessionStorage) {
throw new Error();
}
localStorage.setItem('storage_test', 1);
localStorage.removeItem('storage_test');
Steps to reproduce:
requirejs-bundle-config.js
and balerbundles/core-bundle.js
have no new line in the end.js
files that are not in core-bundle.js
should be minified through tersercreateMinifier
module that will create a pool of minification workers. We'll want to add a minifyFromFile
API, though, so we don't need to read loose files into memory in the main process and then serialize/transfer/parse in the workerHi,
I'm trying baler in a Docker container, and I have infinite spinner with the message (Magento/luma) Create core bundle file
. It looks like the baler does not do anything, and I want to enable a verbose mode or tracing.
This can be done with --trace
CLI argument; however, the filename is generated based on current datetime, and I'd like to be able to specify the trace file name. In my case it's /dev/stdout
.
Please add a parameter with the trace filename.
Console message after "npm run build". Logs are attached. Dependencies were updated, tests were passed well.
@magento/[email protected] build /Users/myName/Sites/baler
tsc
src/createBundleFromDeps.ts:15:28 - error TS2307: Cannot find module './debug'.
15 import { debugEvent } from './debug';
~~~~~~~~~
Found 1 error.
npm ERR! code ELIFECYCLE
npm ERR! errno 2
npm ERR! @magento/[email protected] build: tsc
npm ERR! Exit status 2
npm ERR!
npm ERR! Failed at the @magento/[email protected] build script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
npm ERR! A complete log of this run can be found in:
npm ERR! /Users/myName/.npm/_logs/2019-09-06T07_07_19_540Z-debug.log
2019-09-06T07_07_19_540Z-debug.log
Edit by @DrewML: Bug has been fixed, but a more thorough solution should still be implemented. See later comments in thread for more details
In a multistore setup with different languages, some translations like for "Total" in checkout cart or the ones for the "add to cart" button, just after clicking it, fall back to the ones given for the default store view.
This only happens with the Baler module being enabled.
A colleague of mine debugged it to magento/module-translation/view/base/web/js/image-translation-dictionary.js
This was introduced in the following commit:
magento/magento2@71c781d
Which was seen as a resolution for
#13
I have installed properly baler module on Magento 2.3.4 and its also working but this is not including define or require in bundles and giving errors
mixins.js:5 Uncaught ReferenceError: define is not defined
at mixins.js:5
(anonymous) @ mixins.js:5
jquery.mobile.custom.js:43 Uncaught TypeError: Cannot read property 'event' of undefined
at jquery.mobile.custom.js:43
at jquery.mobile.custom.js:37
at jquery.mobile.custom.js:20
at jquery.mobile.custom.js:11
(anonymous) @ jquery.mobile.custom.js:43
(anonymous) @ jquery.mobile.custom.js:37
(anonymous) @ jquery.mobile.custom.js:20
(anonymous) @ jquery.mobile.custom.js:11
(index):740 Uncaught TypeError: require.config is not a function
at (index):740
(anonymous) @ (index):740
(index):775 Uncaught TypeError: require is not a function
at (index):775
(anonymous) @ (index):775
(index):793 Uncaught TypeError: require is not a function
at
(index):793
We've currently hand-written all the code necessary to find all enabled modules/themes installed in a store.
This was done because we did not want to invoke bin/magento
and incur the bootstrapping cost of Magento core every time baler
is run.
However, @Vinai pointed out to me today that it's just as fast (if not slightly faster) to invoke autoload.php
, and then dump values from the ComponentRegistrar
:
php -r "require 'vendor/autoload.php'; echo json_encode([(new \\Magento\\Framework\\Component\\ComponentRegistrar)->getPaths('theme'),(new \\Magento\\Framework\\Component\\ComponentRegistrar)->getPaths('module')]);"
Switching to this would resolve several open bugs, and allow us to delete several hundred lines of code. In addition, the maintenance burden will be significantly smaller when Magento core makes any changes to the theme/module lookup or registration rules.
Hey there,
Baler hit the following error on the mirasvit elastic search ultimate module:
Error: Unable to locate "module.xml" for the module at path "vendor/mirasvit/module-search-elastic-ultimate"
at findRealModuleRoot (/Users/scottfriend/Sites/baler/src/magentoFS.ts:241:15)
at processTicksAndRejections (internal/process/task_queues.js:89:5)
at getModuleConfig (/Users/scottfriend/Sites/baler/src/magentoFS.ts:204:28)
at async Promise.all (index 154)
at getComposerComponents (/Users/scottfriend/Sites/baler/src/magentoFS.ts:332:18)
at async Promise.all (index 0)
at Object.getComponents (/Users/scottfriend/Sites/baler/src/magentoFS.ts:62:37)
at async Promise.all (index 1)
at Object.collectStoreData (/Users/scottfriend/Sites/baler/src/collectStoreData.ts:15:58)
at Object.run (/Users/scottfriend/Sites/baler/src/cli.ts:20:19)
This composer module doesn't actually contain a Magento Module. It's just a shell of a module to pull in other separate actual Magento modules via the compser.json it contains. Kinda goofy, but I think it allows Mirasvit to sell a module that is really just a bundle of other modules.
The contents of the directory are:
Maybe baler could just skip over modules that don't contain a valid module.xml file?
I tried to run npm test on a Macbook (node v10.16.0, npm 6.9.0), but failed. Part of the log is as follows.
12 info lifecycle @magento/[email protected]~format:check: Failed to exec format:check script
13 verbose stack Error: @magento/[email protected] format:check: `prettier --list-different $(head -1 prettier-globs)`
13 verbose stack Exit status 1
13 verbose stack at EventEmitter.<anonymous> (/usr/local/lib/node_modules/npm/node_modules/npm-lifecycle/index.js:301:16)
13 verbose stack at EventEmitter.emit (events.js:198:13)
13 verbose stack at ChildProcess.<anonymous> (/usr/local/lib/node_modules/npm/node_modules/npm-lifecycle/lib/spawn.js:55:14)
13 verbose stack at ChildProcess.emit (events.js:198:13)
13 verbose stack at maybeClose (internal/child_process.js:982:16)
13 verbose stack at Process.ChildProcess._handle.onexit (internal/child_process.js:259:5)
14 verbose pkgid @magento/[email protected]
In Chrome, a preload
tag ends up pushing the priority of a resource ahead of any other, non-preloaded resource. This is detrimental to the FMP time, because CSS ends up prioritized after the preloaded JS.
Note the special treatment of core-bundle.js
, getting its first byte before any other asset
Note: The timeline above shows all resources sharing bandwidth, but devtools is lying. Each resource is getting downloaded exclusively
I verified through chrome://net-export/
that Chrome is signaling to the server that core-bundle.js
should be the first resource sent after the initial HTML Document. Below are the streams for the first few critical assets, in the order the streams were created:
Resource | Stream Weight | Stream ID | Parent Stream ID | Exclusive Bit |
---|---|---|---|---|
Initial HTML Document | 256 | 1 | 0 | TRUE |
core-bundle.js | 220 | 3 | 0 | TRUE |
calendar.css | 256 | 5 | 0 | TRUE |
styles-m.css | 256 | 7 | 5 | TRUE |
styles-l.css | 256 | 9 | 7 | TRUE |
require.js | 220 | 11 | 3 | TRUE |
Looking through HTTP2_SESSION_RECV_DATA
events in the net export log, you can see the real order the server flushed the responses in below. Note that Chrome used the exclusive
bit on every stream, so assets are given 100% bandwidth and downloaded in the order specified between stream weight and priorities.
Real Download Order:
core-bundle.js
and calendar.css
are both given a parent stream ID of 0
. According to the H2 spec, when the exclusive
bit is used and multiple dependencies are added to a parent stream, it should create this tree
parent stream > first dependency > second dependency
Because of this, core-bundle.js
gets downloaded before any of the css on the page, even though the CSS has a higher stream weight. It seems like usage of the exclusive bit on every stream causes stream weights to be ignored, based on my reading of the spec. In particular, these 2 parts:
An exclusive flag allows for the insertion of a new level of
dependencies. The exclusive flag causes the stream to become the
sole dependency of its parent stream, causing other dependencies to
become dependent on the exclusive stream
https://tools.ietf.org/html/rfc7540#section-5.3.1
Streams with the same parent SHOULD be allocated resources
proportionally based on their weight
https://tools.ietf.org/html/rfc7540#section-5.3.2
Since the exclusive
bit on every stream means no 2 resources will share a parent, the stream weight is never used.
IMO, Chrome's handling of H2 priorities here is wrong/buggy. If Chrome knows it's going to assign a higher weight to calendar.css
than to core-bundle.js
, it should be making calendar.css
a dependency of parent 0 before core-bundle.js
, which would result in the JS being prioritized immediately after the CSS, fixing the FMP timing.
Hello,
I see that for every body, it's working but it does not work for me.
I'm on Magento CE 2.3.3. I just followed this documentation https://github.com/magento/baler/blob/master/docs/ALPHA.md .
My result is that bundles are well created. And the core-bundle js is called. But at the same time, I have many other js files which are called.
I don't know what's wrong ... Maybe it's because of custom codes, on what do we need to be careful ?
After uninstalling the conflicting module for testing purposes (see issue #8) and then running baler again I got the following typeError:
TypeError: Cannot read property 'area' of undefined
at Object.bundleThemes ([magento-root]/baler/src/index.ts:24:19)
at Object.run ([magento-root]/baler/src/cli.ts:23:27)
This leads to the following code:
export async function bundleThemes(
magentoRoot: string,
store: StoreData,
): Promise<BundleResult[]> {
const { components, deployedThemes } = store;
const pendingBundleResults: ReturnType<typeof bundleSingleTheme>[] = [];
for (const themeID of deployedThemes) {
const theme = components.themes[themeID]; //<< LEADS TO HERE 2/2
if (theme.area !== 'frontend' || theme.themeID === 'Magento/blank') { //<< HERE 1/2
continue;
}
I'm not unsure of how to debug this but it seems strange that the theme val is undefined, unless the components.themes array doesn't match IDs with IDs in deployed Themes?
Also: I'm running this on a dev-server - should that even work, or do i need to test on production mode?
The output from the CLI right now is not great.
Developer experience is important for adoption, so I'd like to get this done ASAP.
TODO:
--verbose
flag and debug logs)I was able to apple baler on my local instance but I'm struggling to apply it to Magento Cloud. Is there any way to make baler command to be available in the Magento Cloud auto deployment process?
When a theme is bundled, the following artifacts are written to disk:
requirejs-baler-config.js
balerbundles/core-bundle.js
baler-preload-map.json
The bundler works by collecting a list of all locales for a single theme, but only performs analysis and bundling of the first one found on disk.
Right now, other locales do not get bundling files.
This story is complete when the artifacts referenced above are copied to each locale.
Example: Using luma
, with en_US
, es_ES
, and fr_BE
. In this example, bundling will run against luma/en_US
, and then artifacts will be copied to luma/es_ES
and fr_BE
.
On our production environment we don't have the possibility to use node, so I wonder if I can run baler on our development machine (which does run node), and use the results on our production environment..?
I have two store view with different languages: en_US and zh_Hant_HK, after I run "baler build" command, I can only find the bundle file in en_US folder. Seems the regex rule in src/magentoFS.ts not support the locale such as "zh_Hant_HK".
Sorry I have one question, Magento2 is slow and what you made is really useful. We need every piece of benefit which can reduce the page load time.
And our customers are checking every minute the Google pagespeed score.
I could not find excatly when to run Baler?
In Developer mode or production mode?
I saw you instructed static content deploy, so it must run on production mode since there's no need for static content deploy in developer mode...
I made all the installation step by step (baler is working and telling it finished the bundling with success) but still Google pagespeed advice to use Baler instead of standart bundling of magento.
What I'm doing wrong I dont know
When to flush cache, or we dont need to flush cache, or cache flush destroys everything? which is true.
We are magento implementor company and all other developer, no one made successful baler installation until now...
Please give hand my client's site is https://www.stockperfume.com
Thanks in advance.,
baler build --trace
✔ Collected theme/module data 142ms
⠋ (Magento/luma) Create core bundle file⠋ (Etilize/import) Create core bundle file
Optimization is done, but stats have not been implemented in the CLI yet
⠋ (Etilize/import) Create core bundle file
....
(2492.643615990877): Preparing to analyze "MutationObserver" for dependencies
(2496.74079400301): Preparing to analyze "FormData" for dependencies
(2507.8036310076714): Resolving dependency "jquery/jquery.mobile.custom" from ""
(2508.143660992384): Resolving dependency "jquery" from ""
(2508.2522000074387): Resolving dependency "(2482.613610982895): Resolving dependency "jquery" from "jquery-ui-modules/datepicker"
(2482.7034960091114): Resolving dependency "jquery-ui-modules/core" from "jquery-ui-modules/datepicker"
(2482.7716360092163): Preparing to analyze "jquery-ui-modules/slider" for dependencies
(2491.183733999729): Discovered dependencies for "jquery-ui-modules/slider": jquery, jquery-ui-modules/core, jquery-ui-modules/mouse
(2491.22227999568): Resolving dependency "jquery" from "jquery-ui-modules/slider"
(2491.291195988655): Resolving dependency "jquery-ui-modules/core" from "jquery-ui-modules/slider"
(2491.3486459851265): Resolving dependency "jquery-ui-modules/mouse" from "jquery-ui-modules/slider"
(2492.643615990877): Preparing to analyze "MutationObserver" for dependencies
(2496.74079400301): Preparing to analyze "FormData" for dependencies
(2507.8036310076714): Resolving dependency "jquery/jquery.mobile.custom" from ""
(2508.143660992384): Resolving dependency "jquery" from ""
(2508.2522000074387): Resolving dependency "ns" from ""
(2508.3212589919567): Resolving dependency "vmouse" from ""
(2508.3784649968147): Resolving dependency "support/touch" from ""
(2508.493703007698): Resolving dependency "mage/common" from ""
(2508.5813890099525): Resolving dependency "domReady" from ""
(2508.661637991667): Resolving dependency "mage/dataPost" from ""
(2508.7471689879894): Resolving dependency "mage/template" from ""
(2508.830219000578): Resolving dependency "underscore" from ""
(2508.9048919975758): Resolving dependency "Magento_Ui/js/modal/confirm" from ""
(2509.0140880048275): Resolving dependency "mage/translate" from ""
(2509.0956659913063): Resolving dependency "mage/mage" from ""
(2509.1743820011616): Resolving dependency "mage/apply/main" from ""
(2509.274531006813): Resolving dependency "mage/apply/scripts" from ""
(2509.363907009363): Resolving dependency "jquery/jquery-ui" from ""
(2509.44534099102): Resolving dependency "Magento_Ui/js/modal/modal" from ""
(2509.5548349916935): Resolving dependency "text!ui/template/modal/modal-popup.html" from ""
(2509.5758909881115): Resolving dependency "text" from ""
(2509.7167730033398): Resolving dependency "text" from ""
(2509.785418987274): Resolving dependency "text!ui/template/modal/modal-slide.html" from ""
(2509.8036189973354): Resolving dependency "text" from ""
(2510.175595998764): Resolving dependency "text!ui/template/modal/modal-custom.html" from ""
(2510.2254819869995): Resolving dependency "text" from ""
(2510.3711499869823): Resolving dependency "Magento_Ui/js/lib/key-codes" from ""
(2510.4803000092506): Resolving dependency "mage/bootstrap" from ""
(2510.565497994423): Resolving dependency "Magento_Ui/js/lib/knockout/bootstrap" from ""
(2510.691917002201): Resolving dependency "knockoutjs/knockout" from ""
(2510.7790269851685): Resolving dependency "Magento_Ui/js/lib/knockout/template/engine" from ""
(2510.903595983982): Resolving dependency "Magento_Ui/js/lib/knockout/template/observable_source" from ""
normally it says unknown source but I changed it to say ""..
nevermind it was newer version of jquery.mobile.custom.js causing failure
Internal tracking: GITHUB-150
Then I try to run baler in my root Magento project it stuck with such error:
Error: ENOTDIR: not a directory, scandir ' .. /app/code/.DS_Store'
Yes, I use Mac. To reproduce on Linux try to create the same dir with "."
After bundling using bin/baler build
getting TypeError: require is not a function error
attached error log for your reference.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.