Distributed builds about yarn.build HOT 4 CLOSED

Ooh interesting. It almost sounds like you want a way to tell yarn build that something is already built?

Breaking .yarn/yarn.build.json into multiple files is probably overbuilding, as it's structure is used internally as part of the build tracking. So we'd have to scan the directories, and write out multiple files.

The build command itself does checks on if a run is needed, such as here https://github.com/ojkelly/yarn.build/blob/trunk/packages/plugins/plugin-build/src/commands/supervisor/index.ts#L441

And the plan function manages finding and determining what needs to be built and in what order https://github.com/ojkelly/yarn.build/blob/trunk/packages/plugins/plugin-build/src/commands/supervisor/index.ts#L365

I could build all dependencies in one job and then run separate app build jobs in parallel if I pass build artifacts and yarn.build.json

Note: if you have sufficient concurrency in your CI build this is precisely what yarn.build does. So, you might not need to do anything if you have 2 CPU's available in your CI.

from yarn.build.

Hey @ojkelly! Thank you so much for the quick response!

Do you think it makes sense to add an option so that we could use folderConfiguration info and build or skip by comparing the modified date of the source vs. output folders instead of the manifest (or if the manifest doesn't exist)?

Or maybe simply a --skipIfOutputExists CLI option for build that would simply check if the output folder exists in CI setups.

from yarn.build.

I think --skipIfOutputExists for build is a good addition to solve for this.

I am still a bit curious why you can't build everything once, and then split off and do other tasks with the results. Mainly because it breaks the soundness of the whole solution.

For example if you run yarn build in the top directory, it will determine the optimal plan to build everything, and parallelise it as much as possible.

from yarn.build.

I'm going to close this issue, but feel free to re-open or comment if you are still having trouble or have more information.

from yarn.build.