So, in conclusion, our design goals are:

• Web support first, limited Node.js support (only for SSR with low priority) with limited interoperability
• Bundles, not bundless, generate limited number of chunks
• Using own module system, natively supports ESM, CommonJS modules, other module system can be supported by plugins
• Providing runtime, polyfill to support legacy browsers, but minimum support browsers need to be decided later
• Providing WASM/JavaScript plugins, for better cross-platform support and lower cost for adopting existing plugins
• Making a balance between FP and non-FP in plugin system, for better developing/debug experience and better stability
• Progressive enhancement for new browsers, not graceful degradation

from farm.

After a serious thought, I do not think being compatible with rollup/vite plugins is good ideas any more, below is what I thought:

• Rollup plugins are designed for JavaScript, does not support first class citizen for other resources. This will limit our design if we choose to be compatible with rollup, we will meet the same issues. If new designs supported, we will never be full compatible with rollup plugins.
• Rollup plugins are all written in javascript and do not native support filtering, it is not meet our Performance First Goal

Anyhow, I think we just need to support rollup style plugin, similar plugin hook but not 100% the same. what do your think @roland-reed

Agree. I prefer WASM plugin first. For JavaScript plugins, we can only support rollup-style plugin hooks, but without full-compatible guarantee with existed rollup plugins.

from farm.

Preface

As the web project scales, building performance has been the major bottleneck, a web project compilation using webpack may cost 10min or more, a hmr update may cost 5s or more, heavily reduced the efficiency.

So some tools like vite came out, but vite is using native esm and unbundled, but the compatibility and the huge numbers of module requests becomes the new bottleneck, so methods like caching may improve loading performance but I think It is not good solution, it's just a workaround. And vite is so fast as it uses esbuild which is written in go, which takes performance advantages of the native.

Actually we can take advantages of both webpack and vite, and avoid all of their disadvantages. Webpack is slow, we can use system level languages like rust to greatly improve building performance; Vite is unbundled which means the caching can be finer than webpack, but it has problems like inconsistence(dev and prod) and huge requests numbers, we can using some module merging strategy to reduce the request numbers without losing cache granularity.

As we discussed above, Farm is a web building tool aim to be faster(both building performance and resources loading performance) and more consistent, take advantages of existing tools and discard their disadvantages. But Farm is not aim to be a universal bundler, we just focus on web project compiling, which means our inputs are mainly web assets html, js/jsx/ts/tsx, css/scss/less, png/svg/... and so on, and every design we made will be browser first.

Design Philosophy

• Performance first: Everything will be written in rust as long as we can, only several parts which is not the performance bottleneck will be written in JS
• Rollup style plugin system: Easy to create your own plugins and easy to migrate your plugins/projects from rollup/vite.
• first class citizen support of all web assets: We won't need to transform everything to Javascript any more, we treat anything as first class citizen, in the farm core basic asset like html, js/jsx/ts/tsx, css/scss, png/svg/... will be support by default, you can using plugins to support more assets or transform to the supported assets.
• browser first: Farm's production aims to run in the browser/nodejs(only for SSR), we won't be a universal bundler and only try our best to improve web performance and efficiency.
• Unbundled first: We only merging modules together when the module numbers of size reach our limits, when merging modules we will use a powerful merging strategy to control the resources request numbers without losing cache granularity.
• Consistence first: We will make sure the development and production exactly the same by default, what you see in development will be the same as what you got in production.
• Compatibility: Farm will work with both legacy and modern browser.

Terms

• Module: Basic compilation unit, it may be a file or a virtual module, for example, all kinds of web asset like js, ts, jsx, tsx, css, scss, png, svg..., or virtual modules implemented by plugins.
• Resource: A final generated farm generated after compilation, it may be a js/css/html/png/svg file and may contain may original modules
• ModuleGroup: All static imported modules from an entry will be in the same ModuleGroup.
• ModuleGraph: Dependency graph of all resolved modules
• ResouceGraph: Dependency graph of all generated resources
• ModuleBucket: A collection of modules that will be always together, which means the modules in the same ModuleBucket will aways in the final Resource

Architecture

TODO

Configuration

Our configuration will be designed User First, which means the users can easily find what they want. The configuration is meant to divide into two parts: compilation and server.

Data Structures

TODO

Plugin System

TODO

Module Merging

TODO

from farm.

Design goals

A few decisions about design goals should be make:

• If we are gonna support Node.js (for SSR), are we going to generate IIFE bundles instead of CommonJS/ES Module modules which have better interoperability?
• Are we gonna support legacy browsers, like IE 11, IE 10 etc? If we are, how old are we gonna support?
• What module spec are we gonna use? Use our own module spec, ES Module, CommonJS, System.js or some/all of them?
• For different browsers, which strategy should we choose? Progressive enhancement or graceful degradation?
• Can we introduce FP (functional programing) in our plugin system to reduce uncertainty in plugin development?

from farm.

• I think we may have three kind of module targets: browser, commonjs and esm in the core plugins. The browser is using our module system for supporting legacy browsers. commonjs and esm will wrap our module systems and reexport using commonjs or esm, I think it may be enough for web(browser and nodejs). IIFE and other module target like SystemJs maybe support by community plugins but it is not our first goal.
• yes, I think compatibility is one of goal. We will provide legacy browser support and polyfill.
• using our own module system runtime internally, I think it's will be easier for module merging and hmr and future runtime optimization like resources preload
• Progressive enhancement I think
• yes, we can reduce side effects of each hook but it may be hard to using pure FP

from farm.

• If we use our own module system internally, will there be some performance issues when interoperate with other runtime (I guess)?
• And, are we going to provide multiple chunks for Node.js bundles or only one bundle? I think SSR support has a lower priority, we only need to leave an API
• If we are going to leave System.js/IIFE and other module systems to the community, how can we make sure all module systems have exactly same behavior in runtime?
• I think we should provide more support for new browsers, for example, ESM support in new browsers, our own module system in old browsers, or we may not able to use the strengths of new browsers
• Apart from JavaScript plugins, should we support binary plugins to reduce core size and better extensibility?

from farm.

• webpack is using the same way, I think it is not a big issue, but if we do not use runtime system we will have issues after merging module like cyclic dependency and module orders(rollup and vite has this issue as they do not provide runtime)
• multiple chunks, runtime may be a single chunk, and we only make the production executable on node at beginning.
• other module systems are not the minimal requirements for a web project, I think it has lower priority
• esm is not friendly for hmr after module merging, if multiple module merged into a single esm, how can we update only one of the modules, and how can we maintain the module dependencies across merged js files?
• yes, I am thinking wasm based plugins, which is used now. Wasm plugin can be portable and efficient enough.

from farm.

Theoretically, we use ESM in chunks for production, so that we can provide native support for features like modulepreload, which can provide recursive preload for modules, reducing final HTML size, but since we will only generate limited chunks, the improvement maybe trival.

from farm.

More details will be filled later

from farm.

After a serious thought, I do not think being compatible with rollup/vite plugins is good ideas any more, below is what I thought:

• Rollup plugins are designed for JavaScript, does not support first class citizen for other resources. This will limit our design if we choose to be compatible with rollup, we will meet the same issues. If new designs supported, we will never be full compatible with rollup plugins.
• Rollup plugins are all written in javascript and do not native support filtering, it does not meet our Performance First Goal

Anyhow, I think we just need to support rollup style plugin, similar plugin hook but not 100% the same. what do you think @roland-reed

from farm.

closed due to all rfcs moved to independent rfcs repository

from farm.