Comments (4)
Hi @dschaaff,
I have had a look at the PSI stuff and here is some of my thoughts towards it.
The PSI accounting is not without overhead. That is the main reason why the kernel provides the ability to disable PSI by default through the mentioned config option. Extra code is run at every sleep/wakeup code paths of the scheduler, which seems scary and hints at default disable is the right way to go. However, As far as I understand the impact of that overhead is of an academic nature and can be seen on micro benchmarks but has yet to be proven to impact real-world workloads.
With all of these trade-off it comes down to some preferring one option over the other. With default disabling we are on the safe side of not introducing overhead to those that may be paranoid and do not use PSI, but are making the lives of those that want to use it a bit harder. The other way around we may make your life easier, but need to have others to disable it explicitly. So which side do we pick?
That being said more recent developments have also included some optimizations for the PSI code that recouped some of the overhead and made these code paths more efficient. With our variants that ship with kernel 6.1 we are in fact not disabling PSI by default. You can use it today without the overhead of the extra reboot by using variants aws-ecs-2*
, *-k8s-1.28*
as well as the *-dev
variants. Please let us know if that is an option for you to unblock your use-case.
In the meantime I will have a look at the overhead on the older kernel versions and if the improvements may have been made available to the 5.10 and 5.15 kernel series. That may inform a decision on changing the default for variants with these kernels.
from bottlerocket.
@dschaaff Thanks for opening this issue! We will research on it and reach back to you. Thanks.
from bottlerocket.
Thanks for the thorough response! We've found the pressure stall metrics valuable for a number of database workloads we run on ec2 directly. I'm glad to hear it is enabled for variants with the 6.1 kernel. We are currently on Kubernetes 1.27 with plans to upgrade to 1.28 beginning of next year. I'm content waiting until then to get the pressure stall info. If it happens to get enabled on the 5.15 kernel versions I'll consider that a treat for me :)
from bottlerocket.
So I have had a look at the improvements that were introduced with the 6.1 kernel through this pull request. Given that the improvements are non-trivial in the 4-9% range and these patches have not been back-ported to any earlier series, I am quite opposed to just enable it by default on the older kernels.
Even though, I have not found reports of the impact of older PSI implementations on real-world workloads, there may be users that are sensitive to these kind of things and that would be in for an unpleasant surprise when just doing a minor update. So we will not change the default for 5.10 and 5.15 based variants to be on the safe side.
from bottlerocket.
Related Issues (20)
- Bottlerocket merging bootstrap_extra_args are adding extra quotes when using the official eks terraform module HOT 7
- New setting under `settings.container-runtime` for configuring (`stargz`/`soci`) snapshotter for lazy image pulling HOT 2
- Add additional ECS configuration values to `settings.ecs` HOT 3
- ecr-credential-provider: use custom AWS_PROFILE HOT 4
- Bottlerocket node intermittently fails to start with "[FAILED] Failed to start Wait for Network to be Configured." HOT 11
- v1.19.0 update CHANGELOG
- Remove metal and vmware k8s 1.24 variants by Feb 2024
- v1.19.0 🦍 Tracking Issue HOT 1
- NodePort services inaccessible/blocked by iptables HOT 21
- Missing cAdvisor metrics HOT 2
- Setting to control bottlerocket host cgroup cpu allocation HOT 1
- v1.19.0 update eni-max-pods mapping file
- v1.19.0 Host container updates HOT 1
- v1.19.0 Go dependency updates
- Sandbox container image being GC'd in 1.29 HOT 8
- Specify autoloaded kernel module options via settings. HOT 4
- Update ECS agent to v1.81.0 and Docker to v25
- update to glibc 2.39
- v1.19.1 💘 Tracking Issue HOT 2
- Issue with Bottlerocket image HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from bottlerocket.