Comments (12)
[Tackling only one bit out of the larger context]
In particular, what we really want is some sort of message from the VM that it has entered the initramfs, but before we start processing Ignition.
This sounds not very different from Azure boot check-in, which we are performing in a non-homogeneous way today (RHCOS does it in initramfs, FCOS does is after boot-complete.target
).
We could think about consolidating it in a "first-boot initramfs reached" check-in across various platforms in Afterburn, to be run before Ignition.
However:
- this places a lower bound in the initramfs service ordering, somewhere after the network is up
- the lowest common denominator across clouds (i.e. Azure, Packet, qemu) seems to be a one-shot signal (i.e. a marker, not a rich streaming log)
from afterburn.
Additional note: on Azure the firstboot check-in also ejects the Virtual CD (paging @darkmuggle for confirmation), so I fear we cannot really check in before Ignition fetching is completed.
from afterburn.
(forwarding some real time discussion here)
I think you're absolutely right - we should think of this as "add a first-boot checkin to our qemu model", since that model already exists on some clouds.
And then further discussion turned up https://wiki.qemu.org/Features/GuestAgent - so we could implement the minimum there in Afterburn, and have coreos-assembler time out if the guest doesn't reply to a sync pretty early on.
this places a lower bound in the initramfs service ordering which is somewhere after the network is up
We can make the afterburn checkin not require networking on qemu, but I'm not very concerned about this TBH because qemu networking is quite fast.
from afterburn.
OK, moved this issue to afterburn.
I took a quick look at implementing this. I'd like to propose that we have afterburn run itself as a systemd generator early on startup, rather than shipping static units; this would give us a clean way to order our guest startup unit invocation
After=/dev/virtio-ports/agent
(And we could also tweak the unit to not be After=network-online.target
for the qemu case, etc.)
from afterburn.
And then further discussion turned up https://wiki.qemu.org/Features/GuestAgent - so we could implement the minimum there in Afterburn, and have coreos-assembler time out if the guest doesn't reply to a sync pretty early on.
That doesn't look like a great fit. In particular, the protocol is unidirectional (host->guest) which means that we have to sit in initramfs waiting to be polled (instead of actively signaling a guest->host event, like we do on Azure and Packet), and we can't really leave the initramfs until we have been polled.
The amount of time we should be sitting there is somehow arbitrary, I guess. In the vast majority of cases there won't be anything polling, so we would be now delaying most instances first boots on qemu.
Perhaps a more suitable protocol to target would be the ovirt-guest-agent one, which seems to support sending guest->host events. Conveniently, it already defines a system-startup event.
from afterburn.
You're right, once I started working on the code I noticed the inversion of control.
Discussing the oVirt protocol though gets into the much bigger topic of whether we want to try to implement more of the protocol for real as an agent on that platform, and how the platform would behave with our likely-to-be-a-subset of the functionality.
I guess as a start we could just respond to the channel on ignition.platform.id=qemu
and sidestep that though.
from afterburn.
Additional note: on Azure the firstboot check-in also ejects the Virtual CD (paging @darkmuggle for confirmation), so I fear we cannot really check in before Ignition fetching is completed.
Right, this comment proposes making our systemd units platform-dependent.
(Also we discussed a while changing Ignition on Azure to save the config to /boot
or so)
from afterburn.
Right, this comment proposes making our systemd units platform-dependent.
We don't need to do it as a generator, though, right? We can just ship some static units with ConditionKernelCommandLine=ignition.platform.id=X
.
from afterburn.
We don't need to do it as a generator, though, right? We can just ship some static units with ConditionKernelCommandLine=ignition.platform.id=X.
Yeah; the duplication there might get ugly but OTOH I guess we could generate the units statically i.e. as part of the build process.
from afterburn.
@cgwalters pointed me to this ticket, so for the record, as he knows, in the last week I've been working on running openQA tests on Fedora CoreOS. It's not very difficult, and we have it working already, the only question mark for a 'production' deployment would be when and on what to trigger the tests.
openQA definitely does do a fairly good job of letting you know if the artifact under test boots successfully in a VM.
For now the work is living on a branch of the openQA test repo and is only deployed on my pet openQA instance which is not up all the time (it heats up my office...:>), we can do a production deployment quite easily once the triggering questions are sorted out.
from afterburn.
openQA definitely does do a fairly good job of letting you know if the artifact under test boots successfully in a VM.
To be clear kola already covers this pretty well in general - we just have a few specific gaps, such as the case when a Secure Boot signature validation fails.
from afterburn.
for the record once more, we did deploy the openQA CoreOS testing stuff to production. The scheduling works by just checking once an hour if any of the named streams has been updated, and scheduling tests for the new build if so. Results show up at https://openqa.fedoraproject.org/group_overview/1?limit_builds=100 , e.g. here. We can write/run more tests if desired, requests can be filed at https://pagure.io/fedora-qa/os-autoinst-distri-fedora/issues .
from afterburn.
Related Issues (20)
- Release Afterburn 5.5.0 HOT 1
- Release Afterburn 5.5.1 HOT 3
- Release Afterburn 5.6.0 HOT 4
- Latest MSRV lint test has found some dead_code sections
- Release Afterburn 5.7.0 HOT 3
- Test `providers::aws::mock_tests::test_aws_basic` is flaky HOT 2
- Test `providers::aws::mock_tests::test_aws_imds_versions` is flaky HOT 3
- Support IPv6 on AWS HOT 1
- Release 5.3.0 HOT 3
- afterburn-sshkeys fails if no ssh key in metadata on ibmcloud HOT 1
- Consider switching argument parsing to `clap_derive`
- Consolidate CloudStack metadata providers
- Consider enabling hostname service on subsequent boots
- Release Afterburn 5.4.0 HOT 2
- Release Afterburn 5.4.1 HOT 1
- configure user account based on azure provider specific information HOT 1
- Release Afterburn 5.4.2 HOT 7
- Afterburn reports incorrect OpenStack instance id HOT 2
- Release Afterburn 5.4.3
- Move away from `users` crate HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from afterburn.