Comments (10)
Unpacking platform in sparse mode
Loaded extensions information from project custom folder in 39 ms
Loaded extensions information from hybrisPlatform dependencies in 2628 ms
Loaded all needed extensions from localextensions.xml in 1 ms
Loaded existing extensions information from project folder in 655 ms
No missing SAP Commerce extensions, nothing to unpack
This is from 2211.2 with intextpack 2211.3, on an M1 Max. With these numbers in mind, I do not believe we need any caching of extension info. I would make the PR as is and we can add caching if needed.
from commerce-gradle-plugin.
This is the run on a Windows machine:
Unpacking platform in sparse mode
Loaded extensions information from project custom folder in 160 ms
Loaded extensions information from hybrisPlatform dependencies in 10539 ms
Loaded all needed extensions from localextensions.xml in 2 ms
Loaded existing extensions information from project folder in 107 ms
Some needed SAP Commerce Suite extensions are missing, copying them
It is slower, but still good enough to not require any extra optimization/caching.
from commerce-gradle-plugin.
Thanks for reporting the issue!
I'm using Gradle / Java to delete the files/folders to be OS-agnostic, but you are definitely right about the performance.
Because now that you mention it, I had the same issues when implementing cleanPlatform
Long story short: project.delete
is not fast enough, need to either use native Java or OS calls like you mentioned :)
from commerce-gradle-plugin.
How about, instead of cleaning up the unused extensions, we extract from the platform zip only the ones needed by local extensions.xml (and dependencies)? If we could improve the extractions of the platform in that direction, I can help with the implementation.
from commerce-gradle-plugin.
Thats sounds imteresting!
I guess you would:
build the complete dependency tree by parsing all extensioninfo.xml
files across all zips (dependency configuration hybrisPlatform
), resolve it against localextensions.xml
and based on the pruned tree, unpakck only what's necessary.
from commerce-gradle-plugin.
Something along the line:
- phase 1: gather information about all extensions and their direct dependencies
- gather information about all extensions in the custom folder by parsing the
extensioninfo.xml
files in there and building a list of objects describing each extension and its direct declared dependencies (e.g.ExtensionInformation
) - gather information about all extensions in the commerce suite zip (obtained from project dependency configuration
hybrisPlatform
) by parsing theextensioninfo.xml
files in the zip and build also for them a lost of extension objects - repeat the above step also for the optional integration extension pack (if it is declared)
- add the
platform
dependency as a must (a single extension for the entire platform should be enough, not each individual extension inplatform/ext
, as the platform entirely should be always present
-phase 2: build the list of needed hybris dependencies to be present - add the platform extension
- for each extension declared in
localextensions.xml
, add it in the needed list, then add its dependencies, recursively
- gather information about all extensions in the custom folder by parsing the
- phase 3: build the list of extensions already present in the hybris
bin
dir (not in the zip, but the project folder) - phase 4: find the missing extensions, by removing from the needed list the present ones
- phase 5: extract from the zips (hybris commerce suite and integration pack) the missing extensions
Since the entire parsing of all the extensioninfo.xml
files in the commerce suite zip and intpack zip can take some time, to make the next builds faster, we can save the resulting list of extension information objects in a file in the project cache directory and use that file as long as we do not need a different version of the commerce suite or intpack.
As soon as the version of the commerce suite dependency or of the intpack dependency changes from what we saved in the project cache, we proceed to delete the entire hybris bin/modules
and bin/platform
folders and start over with the steps above.
Another nice bonus we can have is to unzip some "always needed" extensions (e.g. yempty
, yocc
, ybackoffice
), that are handy to have for ant extgen
but we do not want them declared in the localextensions.xml
of any project. We can configure these as a list property of our plugin.
What do you think?
from commerce-gradle-plugin.
I like both, the idea and the proposal. Do we have some dry runs to determine what the actual effect on the build time really is? (Just interested in it)
Besides the decrease of build times it will also safe disk space, which i also like.
You got my vote up 👍
from commerce-gradle-plugin.
I also like both the idea and the proposal, feel free to go ahead and submit a PR!
A few additional remarks:
- the new behavior should be opt-in
- this idea deserves its own section in the plugin config
- tests!!
hybris {
sparseBootstrap {
enabled.set(true)
alwaysIncluded.set(listOf("yempty", ...))
}
}
(Disclaimer: I just quickly typed this up. Suggestions for better names, different structure, ... are always welcome)
from commerce-gradle-plugin.
Good, then I will implement this in between next week and we will see how it goes.
from commerce-gradle-plugin.
closed in favour of sparseBootstrap
(#43)
from commerce-gradle-plugin.
Related Issues (20)
- CCv2 plugin: allow comments in manifest.json HOT 1
- Add dependency to bootstrapPlatform for tasks created by HybrisAntRule HOT 3
- Cloud tests failure should fail the build by default
- DefaultTask is created for cloud tests when no test configuration is defined
- Automating downloads from launchpad with CI HOT 8
- Improvement : Pass ant property via command line HOT 1
- Investigate slow cleanPlatform w/ Gradle 7.4 HOT 3
- Support for SAP Commerce 2105 HOT 3
- HybrisAntTask xPath error in new SAP commerce version 2205 HOT 4
- manifest validation fails for 2211
- Refactor mpern.sap.commerce.build.util.Extension HOT 1
- Gradle 8 support HOT 2
- Spring Cleaning 2023 HOT 1
- sap.commerce.build 3.9.0 not compatible with 2211.7 (FP1) HOT 5
- Sparse build slow when building smartedittools HOT 3
- Error with default excludes and gradle >= 6.7 HOT 1
- plugin 3.9.1 always extracts 2211.8 from scratch HOT 2
- Mismatched Image Version Validation Issue. HOT 1
- HybrisAntTask.setMain() fails for yclean HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from commerce-gradle-plugin.