Comments (3)
Thanks, @joshfriend, we'll get back to the this issue soon, I just wanted to clarify one point:
We are able to run task execution in parallel with CC turned off, I don't understand why this is a requirement when CC is enabled.
By isolating tasks, CC can enable intra-project parallelism while --parallel
only enables inter-project parallelism.
from gradle.
Can you tell us more about what you perceive as a source of the overhead? We know about sequential dependency resolution, and will address this soon. Is the I/O in general a problem? Do you expect the machine to have enough memory to temporarily hold the cached state if needed?
The store
operation is an essential step to prepare for the execution phase, to allow parallel task execution, for example. It doesn't really happen at the end of the build. Our end goal is to get rid of non-configuration-cached execution, so falling back to it won't be a long-term solution.
An alternative solution could be to fail the build if the cached state cannot be reused to indicate that cache-priming build has to be re-run. Does this behavior fit your use case?
from gradle.
Can you tell us more about what you perceive as a source of the overhead?
At one point, we had enabled configuration cache in our CI as a way to validate that our build was compatible with CC when updating gradle, but we found that this came at a ~10% performance penalty. For larger builds we would sometimes observe the storing of configuration cache to take >1m.
We are rolling out configuration caching to CI builds where we produce the cache in the main
branch, and PR builds will restore the cache from the nearest commit ancestor on main
that has cache available. In some cases, a developer has made a CC invalidating change and the cache we restore is not reusable. In these cases we would like to basically continue as if CC were disabled and not incur the cost of storing the new configuration to the cache.
Do you expect the machine to have enough memory to temporarily hold the cached state if needed?
Generally yes, we had one or two CI jobs in the build where configuration cache had to remain disabled because it caused OOMs, but we have been able to hold the state in memory for everything else.
The store operation is an essential step to prepare for the execution phase, to allow parallel task execution, for example
We are able to run task execution in parallel with CC turned off, I don't understand why this is a requirement when CC is enabled. I think I am missing some bit of knowledge here that would help this requirement make sense.
An alternative solution could be to fail the build if the cached state cannot be reused to indicate that cache-priming build has to be re-run. Does this behavior fit your use case?
Potentially,. We would have to check if the time taken to run the initialization twice with different settings would be faster than writing the configuration cache and discarding it. That doesn't seem great in general from a usability standpoint though.
from gradle.
Related Issues (20)
- Dependency verification failed after just invoked write-verification-metadata HOT 2
- Untangle Service Registry framework implementation
- Project.findProperty does not search through this project's ancestor projects. Instead, it skips right to the root if there is no property local to the current project HOT 4
- TestLauncher does not discover tests in Junit5 @Nested class HOT 1
- Add test documenting the failure of #29176
- buildSrc plugin cannot use more than one repository HOT 4
- request more accurate information when emitting dependencies arising from gradle plugins HOT 1
- Option `gradle-user-home` changes its behavior depending on position in command line HOT 1
- When wrapper fails checksum validation, don't show the actual checksum, instead redirect users to a more reliable source HOT 2
- Secondary variants break consumable configuration with no artifacts HOT 1
- Gradle public APIs to get repository info HOT 1
- A build scan cannot be produced due to "Dangling build operations detected".
- Configuration cache should not reduce `map`-ed TaskProvider value
- `@OutputFile RegularFileProperty` to `@Input Provider<String>` throws FileNotFoundException with the configuration cache on HOT 1
- Binary compatibilty reports method not removed for upgraded method if method is removed in a class and it's super class
- Built-in task to encrypt Gradle properties HOT 2
- DependencyContraintHandler is missing addProvider methods HOT 1
- Applying Gradle plugin from included build fails when the plugin published by the build is applied to multiple projects HOT 4
- Support Grade working in environments that only support IPv4 or IPv6 networks
- 8.5 thru 8.8 busted on jdk-22. HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from gradle.