webpack / loader-runner Goto Github PK
View Code? Open in Web Editor NEWRuns (webpack) loaders
License: MIT License
Runs (webpack) loaders
License: MIT License
Introduces getOptions
function inside loaderContext.
The implementation of this function should mirror the one inside webpack NormalModule.
I think we can use loader context in the same way of webpack to extract loader's options and validate them if a schema is provided (Include schema-utils
and json-parse-better-errors
(or maybe directly use JSON.parse) as dependencies).
getOptions
will become available to loaders called via loader-runner
.
thread-loader
is one of them and has an open issue about undefined this.getOptions
.
I have created a simple reproduction repository for reference.
If you agree I can try to do a PR.
Hi, I've run into an interop problem between systemjs-builder and webpack running concurrently in the same process. In short, loader-runner picks up on the System
global installed by systemjs due to this code path.
I don't know if it's really loader-runner's fault, but I ended up solving my problem by forking it and cutting out that part. And now that webpack 2 went with import(...)
instead of System.import(...)
, is it still needed?
Hi, I'm running into this issue when trying to load a file from inside the loader:
`ReferenceError: fs is not defined
35 | console.log(this.getDependencies());
36 |
> 37 | fs.readFile(fileToSearch, 'utf8', (error, data) => {
| ^`
Error doesn't happen when running from webpack config. Only with loader-runner.
Could you help me out?
Would be great if you could release a new version. webpack/webpack#3180 is quite annoying bug and it is already fixed in master.
I use the lib to test my custom loader, and my loader depend on context.exec
.
So, is there a plan to support context.exec
in the future?
I'm trying to debug an issue I'm having with System.import
within loader-runner right now, but honestly I'm not sure where System
is even coming from in the first place. It's not part of this module, or webpack core.
The issue is that when it's given a path to import, it is unable to correctly resolve require
s or imports
from inside that file. Confusingly, it tries to resolve from the root of the webpack project, not even any node_modules
folder. So for example, I have something like this file being System.import
ed:
const someModule = require('some_module')
module.exports = function someLoader (source) {
return source
}
When you run System.import
with the path to this file, it throws an error from trying to resolve some_module
relative to the project root, where it certainly does not exist. It seems like it resolves it according to its baseURL
property, which is set as the project root.
It seems like it comes from SystemJS, the System
object is called SystemJSNodeLoader
, and defines its own version as 0.19.31 Node
.
If you pass loader options that have a function within them, something like
{
loader: 'css-loader',
options: {
url: {
filter:
(url) => {
if (url.includes('abc')) {
return false;
}
return true;
}
}
}
}
then this will be not converted to a function in the worker. Instead the values are missing.
Simply JSON.stringify is called, which returns an empty object for url.
Instead, we should go through each child and if it is a function we need to call toString.
Later in the worker, we then need to set an eval + stringified function, so that it gets set correctly.
Hello, is it possible to incorporate the LICENSE file inside the repo and npm package ? Now it's missing. It's implied by MIT license that if licensed under the MIT the source code must be accompanied by full license text. If the license file is not in the repo nor npm package and lot of people cannot use it.
Thank you
We've already provided a PR solving this: #27
@ChALkeR, @TheLarkInn, @sokra could you help us pls or anybody from your webpack org ?
It would be great if there was a way to store some meta data related to the compiled resource across the whole stack of loaders.
I'm trying to parse front-matter early on to then use the collected values later for generating extra source a couple of loaders later. Unfortunately, the meta
argument I pass through this.callback
gets lost after the next loader.
Would you be open to having the loaderContext
used to invoke each loader bear an extra property (resourceMeta
?) where loaders could store metadata to keep during the run?
This is your team's own problem, you don't know? you have no tests?
@webpack team
Hey, looking to contribute to this! Does any of this codebase require refactoring to ES6?
Given the following config(in file webpack.config.js) using either ts-loader or awesome-typescript-loader:
const path = require('path');
let _root = path.resolve(__dirname);
var fromRoot = function (...args) {
args = Array.prototype.slice.call(args, 0);
return path.join.apply(path, [_root].concat(args));
};
module.exports = {
context: fromRoot( "src" ),
devtool: "cheap-module-eval-source-map",
entry: {
"app/polyfills": "./app/polyfills.ts",
"app/vendor": "./app/vendor.ts"
},
resolve: {
modules: [
fromRoot( "src", "node_modules" ),
fromRoot( "node_modules" )
],
extensions: ["*", ".js", ".ts"],
alias: {
build: fromRoot( "build", "src", "public" ),
jquery: fromRoot( "src", "node_modules", "jquery", "src", "jquery" )
}
},
module: {
rules: [{
test: /\.ts$/,
use: [
{
loader: "ts-loader",
options: {
configFileName: fromRoot( "src", "tsconfig.json" ),
compilerOptions: {
rootDir: fromRoot( "src" ),
outDir: fromRoot( "build", "src", "public" )
}
}
}
]
}
]
},
output: {
path: fromRoot("build", "src", "public"),
publicPath: "/",
filename: "[name].js",
chunkFilename: "[id].chunk.js"
}
};
export = webpackMerge(commonConfig, {
devtool: "cheap-module-eval-source-map",
output: {
path: fromRoot("build", "src", "public"),
publicPath: "/",
filename: "[name].js",
chunkFilename: "[id].chunk.js"
}
}, webpackDevConfig);
Running webpack
works.
Given the following file, test.js:
var webpack = require("webpack");
var webpackConfig = require("./webpack.config.js");
var compiler = webpack(webpackConfig, (err, stats) => {
if (err) {
console.error(err);
};
});
Running node ./test.js
works.
But given the following gulp task:
var webpack = require("webpack");
var webpackConfig = require("./webpack.config.js");
var starting = true;
gulp.task("ugh", function(cb) {
var compiler = webpack(webpackConfig, (err, stats) => {
if (err) {
console.error(err);
};
if (starting) {
starting = false;
cb();
}
});
});
Running gulp ugh
yields these errors:
Unhandled Rejection at: Promise Promise {
<rejected> TypeError: Cannot read property 'default' of undefined
at /Users/mitchellmorris/Sites/projects/sandbox/node_modules/loader-runner/lib/loadLoader.js:4:66
} reason: TypeError: Cannot read property 'default' of undefined
at /Users/mitchellmorris/Sites/projects/sandbox/node_modules/loader-runner/lib/loadLoader.js:4:66
Unhandled Rejection at: Promise Promise {
<rejected> TypeError: Cannot read property 'default' of undefined
at /Users/mitchellmorris/Sites/projects/sandbox/node_modules/loader-runner/lib/loadLoader.js:4:66
} reason: TypeError: Cannot read property 'default' of undefined
at /Users/mitchellmorris/Sites/projects/sandbox/node_modules/loader-runner/lib/loadLoader.js:4:66
From what I can tell given the following block that System(in node_modules/loader-runner/lib/loadLoader.js) only defined when leveraging gulp and emitting an undefined result(i.e. module).
module.exports = function loadLoader(loader, callback) {
if(typeof System === "object" && typeof System.import === "function") {
console.log("This only happens using gulp.");
System.import(loader.path).catch(callback).then(function(module) {
console.log("further more module is undefined!");
loader.normal = typeof module === "function" ? module : module.default;
// ...
});
} else {
console.log("This only happens using node ./test.js or webpack and is successfull.");
// ...
}
}
Can anyone help please?
How do you indicate via config or options that a loader should be loaded using import
instead of require
?
How do you get this logic to kick-in?
I have a loader that is written with ES modules, and my package.json specifies "type": "module"
. I am running Jest using the node option --experimental-vm-modules
and my tests written with ES modules are all working fine except for this one which has the following error: NonErrorEmittedError: (Emitted value instead of an instance of Error) Error: Must use import to load ES Module
.
Relevant stack trace:
at processResult (node_modules/webpack/lib/NormalModule.js:718:12)
at node_modules/webpack/lib/NormalModule.js:827:5
at node_modules/loader-runner/lib/LoaderRunner.js:399:11
at node_modules/loader-runner/lib/LoaderRunner.js:185:11
at loadLoader (node_modules/loader-runner/lib/loadLoader.js:33:11)
at iteratePitchingLoaders (node_modules/loader-runner/lib/LoaderRunner.js:182:2)
at runLoaders (node_modules/loader-runner/lib/LoaderRunner.js:397:2)
at NormalModule.doBuild (node_modules/webpack/lib/NormalModule.js:781:3)
at NormalModule.build (node_modules/webpack/lib/NormalModule.js:928:15)
Your package requires 'fs'. This package name is not currently in use...
Please see:
https://www.npmjs.com/package/fs
Webpack loaders have access to this.resolve
. This is missing in the loader-runner tool.
Hey there!
I was using the lib and discovered that result.result
that I get is an array. But README says:
// result.result: Buffer | String
// The result
// only available when no error occured
I'm not quite sure whether there an error or not, but probably type declaration in the README should be different.
Loaders work if the path contains #
character.
Webpack build fails with ERROR (tested on ts-loader and css-loader):
../../a#/ts-loader-path-issue/src/index.ts 39 bytes [not cacheable] [built] [code generated] [1 error]
ERROR in ../../a#/ts-loader-path-issue/src/index.ts
Module build failed (from ../../a#/ts-loader-path-issue/node_modules/ts-loader/index.js):
Error: Cannot find module 'C:\a#\ts-loader-path-issue\node_modules\ts-loader\index.js'
Require stack:
- C:\a#\ts-loader-path-issue\node_modules\loader-runner\lib\loadLoader.js
- C:\a#\ts-loader-path-issue\node_modules\loader-runner\lib\LoaderRunner.js
- C:\a#\ts-loader-path-issue\node_modules\webpack\lib\NormalModule.js
- C:\a#\ts-loader-path-issue\node_modules\webpack\lib\NormalModuleFactory.js
- C:\a#\ts-loader-path-issue\node_modules\webpack\lib\Compiler.js
- C:\a#\ts-loader-path-issue\node_modules\webpack\lib\webpack.js
- C:\a#\ts-loader-path-issue\node_modules\webpack\lib\index.js
- C:\a#\ts-loader-path-issue\node_modules\webpack-cli\lib\webpack-cli.js
- C:\a#\ts-loader-path-issue\node_modules\webpack-cli\lib\bootstrap.js
- C:\a#\ts-loader-path-issue\node_modules\webpack-cli\bin\cli.js
- C:\a#\ts-loader-path-issue\node_modules\webpack\bin\webpack.js
at Function.Module._resolveFilename (internal/modules/cjs/loader.js:880:15)
at Function.Module._load (internal/modules/cjs/loader.js:725:27)
at Module.require (internal/modules/cjs/loader.js:952:19)
at require (C:\a#\ts-loader-path-issue\node_modules\v8-compile-cache\v8-compile-cache.js:159:20)
at loadLoader (C:\a#\ts-loader-path-issue\node_modules\loader-runner\lib\loadLoader.js:19:17)
at iteratePitchingLoaders (C:\a#\ts-loader-path-issue\node_modules\loader-runner\lib\LoaderRunner.js:182:2)
at runLoaders (C:\a#\ts-loader-path-issue\node_modules\loader-runner\lib\LoaderRunner.js:395:2)
at NormalModule.doBuild (C:\a#\ts-loader-path-issue\node_modules\webpack\lib\NormalModule.js:631:3)
at NormalModule.build (C:\a#\ts-loader-path-issue\node_modules\webpack\lib\NormalModule.js:775:15)
at C:\a#\ts-loader-path-issue\node_modules\webpack\lib\Compilation.js:1236:12
On Windows 10 with run these commands in PowerShell (tested with 5.1):
mkdir C:\a#
cd C:\a#
git clone https://github.com/csutorasa/ts-loader-path-issue.git
cd ts-loader-path-issue
npm install
.\node_modules\.bin\webpack
# if webpack fails set execution policy and run the last command again
# Set-ExecutionPolicy Bypass -Scope Process -Force
On Linux run these commands in bash (tested with 5.0.17):
mkdir -p ~/a#
cd ~/a#
git clone https://github.com/csutorasa/ts-loader-path-issue.git
cd ts-loader-path-issue
npm install
node_modules/.bin/webpack
https://github.com/csutorasa/ts-loader-path-issue (ts-loader repo)
https://github.com/csutorasa/loader-path-issue (css-loader repo)
How to debug this issue, when loadLoader.js crashes on line 4 (module
is undefined
):
Source: https://github.com/kriasoft/react-static-boilerplate/tree/webpack2
When I used thread-loader to speed up compilation, I encountered a bad case on windows. When I refrence the es5-ext package, thread-loader will report the following error:
ERROR in ./node_modules/es5-ext/string/#/ends-with/index.js
Module build failed (from ./node_modules/thread-loader/dist/cjs.js):
Thread Loader (Worker 0)
EISDIR: illegal operation on a directory, read
at PoolWorker.fromErrorObj (D:\Development\test-thread-loader\node_modules\thread-loader\dist\WorkerPool.js:346:12)
at D:\Development\test-thread-loader\node_modules\thread-loader\dist\WorkerPool.js:219:29
at mapSeries (D:\Development\test-thread-loader\node_modules\neo-async\async.js:3625:14)
at PoolWorker.onWorkerMessage (D:\Development\test-thread-loader\node_modules\thread-loader\dist\WorkerPool.js:173:34)
at D:\Development\test-thread-loader\node_modules\thread-loader\dist\WorkerPool.js:146:14
at Socket.onChunk (D:\Development\test-thread-loader\node_modules\thread-loader\dist\readBuffer.js:40:9)
at Socket.emit (events.js:314:20)
at Socket.Readable.read (_stream_readable.js:507:10)
at Socket.read (net.js:625:39)
at flow (_stream_readable.js:1007:34)
@ ./src/index.js 1:0-49 3:12-20
The minimum unit test for this error can be viewed at https://github.com/YourWildDad/test-thread-loader
I think it is because the loader-runner splits the resource fragment, which leads to abnormal reading of the file.
Please help to fix this error, good luck.
See https://github.com/webpack/loader-runner/blob/master/lib/LoaderRunner.js#L126-L128
catch resolve the promise, so then is executed
I have a ES6 angular 13 project that runs angular-cli
with customWebpackConfig
ES6 script:
angular.json:
"customWebpackConfig": {
"path": "./custom-webpack.config.mjs"
},
custom-webpack.config.mjs
import path from 'path';
import webpack from "webpack";
export default function(config) {
config.module.rules.unshift({
test: /app\.ts$/i,
exclude: [/node_modules/],
use: [{loader: path.resolve('./custom-loader.mjs')}]
});
return config;
};
custom-loader.mjs
import _sourceMap from "source-map";
import fs from 'node:fs';
export default function(content, sourceMap) {...}
What is the current behavior?
Error: Module build failed (from ./custom-loader.mjs):
Error [ERR_REQUIRE_ESM]: Must use import to load ES Module: /custom-loader.mjs
at Module.load (internal/modules/cjs/loader.js:935:11)
at Function.Module._load (internal/modules/cjs/loader.js:778:12)
at Module.require (internal/modules/cjs/loader.js:961:19)
at require (internal/modules/cjs/helpers.js:92:18)
at loadLoader (/node_modules/loader-runner/lib/loadLoader.js:19:17)
at iteratePitchingLoaders (/node_modules/loader-runner/lib/LoaderRunner.js:182:2)
at runLoaders (/node_modules/loader-runner/lib/LoaderRunner.js:398:2)
at NormalModule._doBuild (/node_modules/@angular-devkit/build-angular/node_modules/webpack/lib/NormalModule.js:819:3)
at NormalModule.build (/node_modules/@angular-devkit/build-angular/node_modules/webpack/lib/NormalModule.js:963:15)
at /node_modules/@angular-devkit/build-angular/node_modules/webpack/lib/Compilation.js:1371:12
On the other hand, when I add the parameter type: "module"
in the use
array:
use: [{loader: path.resolve('./custom-loader.mjs'), type:"module"}]
I receive an error:
Generating browser application bundles (phase: setup)...An unhandled exception occurred: Invalid configuration object. Webpack has been initialized using a configuration object that does not match the API schema.
- configuration.module.rules[0].use[0] has an unknown property 'type'. These properties are valid:
object { ident?, loader?, options? }
But, if I remove this validation the ES6 script is executing successfully.
Instead of changing Webpack validation schema I also tried to update the LoaderRunner.js
, line 78 and ProgressPlugin.js
, line 405. It also worked:
obj.type = value.type || value.options && value.options.type || '';
It looks like this feature already exists. It just isn't enabled?
Other relevant information:
webpack version: 5.74
Node.js version: 14 and 16
Operating System: MacOS
Additional tools:
@angular-devkit/build-angular: 13.3.9
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.