[5.0.0 osx] Unable to start the podman machine with osx Monterey about podman HOT 15 OPEN

@urkle Podman v4.9 was an experiment in podman machine, and since we were relatively inexperienced in that area, we needed to re-vamp our machine stack almost completely for 5.0. 5.0 to fixes a lot of these potholes, but the gap between 4.9 and 5.0 was big enough that implementing direct upgrades was not possible. Unfortunately, it may have caused a rough transition between 4.9 and 5.0, but, we hope that our fixes in Podman 5.0 and beyond makes our UX much better. Our blog goes into a bit more detail about this.

If you can, we'd recommend you upgrade to Podman v5.0. However, in the case that you're unable to do so, using the following commands to init a new 4.9 machine should fix your issue. Let me know if this gets things working again! And of course, feel free to file any more issues you come across.

x86:

podman machine init --image-path https://builds.coreos.fedoraproject.org/prod/streams/stable/builds/39.20240407.3.0/x86_64/fedora-coreos-39.20240407.3.0-qemu.x86_64.qcow2.xz

arm:

podman machine init --image-path=https://builds.coreos.fedoraproject.org/prod/streams/stable/builds/39.20240407.3.0/aarch64/fedora-coreos-39.20240407.3.0-qemu.aarch64.qcow2.xz

from podman.

@hritik5102 I've had the same problem here and aftering to search I'm found this article:

• Migration of Podman 4 to Podman 5 machines

In the case of Macs running Podman 4, it was based on QEMU virtualization. Podman 5 uses the Apple Hypervisor on Macs and support for QEMU was deprecated. Also, depending on where you have obtained your Podman 5 binaries, you will want to make sure you have vfkit installed as it replaces QEMU.

Although the problem persisted, following the steps outlined in this article the dreaded vfkit exit 1 and removing the brew install to use the Installer Launcher resolved the issue for me.

from podman.

Unfortunately I think the best way forward is to move back to Podman 4.9, by uninstalling it through brew and re-installing it via the pkginstaller shipped on our release page: https://github.com/containers/podman/releases/tag/v4.9.3

from podman.

Thanks for the reply. Do you know if 4.9.X will be supported for a while within podman-desktop or this setup is actually sort of deprecated. Is there a possibility to fix that in podman, is efi really required? Would be nice if it could detect older macs and use a non efi version in the future.

Thanks again.

from podman.

I think that's a question to ask on podman-desktop for desktop support.

@baude @cfergeau Do you know about our EFI requirement?

from podman.

The only alternative to using EFI would be to extract the kernel/inintrd/kernel cmdline from the VM image, and provide these to vfkit. Using EFI is simpler as vfkit/apple virtualization framework can then boot the disk image directly.
EFI support is only available in macOS 13 or newer, but I'm afraid a fallback LinuxBootloader would be quite some work.

from podman.

A friendly reminder that this issue had no activity for 30 days.

from podman.

@ashley-cui how does one exactly get podman 4.9.x working again? as my podman machine auto-upgraded yesterday and switched to podman 5 whcih means I cannot do anything as I can't start a podman 5 machine and I cannot connect the 4.9 client to my qemu machine.

This whole upgrade to podman 5 has not been handled well.

from podman.

@ashley-cui thanks for the detailed information.

I had tried to do a full upgrade to 5.0 but ran into the issue described here and was unable to start the machine.

after much fiddling I managed to get the 4.9.4 installed and used --image-path stable to get the latest 39.x machine created. Then I further disable zincati ( I believe that is all that is needed to prevent auto updates). Further I had to do a lot of "fun" creative work to extract out my volumes from the old image to import into the new image since there is no direct way to export/import volumes in podman machine.

I had to

1. ssh into the vm (before upgrading to podman 5)
2. run podman volume export on each volume (in the vm)
3. use podman machine ssh to "cat" the tar to a local file.
   podman machine ssh cat myvolume_name.tar > myvolume_name.tar
4. then after the failed 5 upgrade and getting the 4.9 vm build back up I ended up using a busybox container to copy the volumes back in. (probably should have just done that to export them as well, but was hoping the export/import would have been nicer.. )

Is there not a possibility of adding back the QEMU backend as an alternative for older macOS?

from podman.

@urkle Hmm interesting, I believe we disabled zincati already with this commit, that's strange that it's still updating. I can take a look at that.

We moved to appleHV as it resolved a lot of issues wrt dependencies, speed, and some other factors. We currently plan on only supporting appleHV for our future.

from podman.

@ashley-cui it could be my original podman machine was created before podman 4.8.0 and that is why it had updated itself?

from podman.

@urkle That would be it!

from podman.

Problem

I was doing Podman machine configuration setup for M1, and when I ran the podman machine start command, It failed with

Starting machine "podman-machine-default"
Error: vfkit exited unexpectedly with exit code 0

Addition information:

• I installed the latest podman version 5.0.2 using brew install podman
• I installed CoreOS version 37.20230401.3.0 instead of 38.20230430.2.1 because the Postgres container crashed during the initial schematic setup on version 38.20230430.2.1, causing the database to be in an invalid state.
• Here are the steps that I've followed.
• Download the older version OS image file (link)

curl -O https://builds.coreos.fedoraproject.org/prod/streams/stable/builds/37.20230401.3.0/aarch64/fedora-coreos-37.20230401.3.0-qemu.aarch64.qcow2.xz

• Initialise the podman machine

podman machine init --disk-size 50 --memory 6144 --cpus 2 --image-path /var/podman_home/fedora-coreos-37.20230401.3.0-qemu.aarch64.qcow2.xz

• Start the machine - podman machine start
• Check the screenshot attached below for more information

Things that i've tried:

• I followed the steps mentioned here and removed the podman machine rm podman-machine-default and reinstall the coreos and did the podman machine configuration again from the start, but it didn't work out.
• Followed this step as well, but it didn't work out: https://podman-desktop.io/docs/troubleshooting/troubleshooting-podman-on-macos#recovering-from-a-failed-start

Note

I'm not sure whether this is relevant or not, I've checked no podman-machine-default file is created inside ~/.ssh folder.

@ashley-cui Could you please help me here? Thank you.

from podman.

JFYI : The above issue got resolved.

Solution:

• Installed the latest version of podman ( viz 5.0.3 ), Not sure, whether this is actually contributed to the fix.
• Installed the latest CoreOS ( i.e fedora image ) version. Instead of using 37.20230401.3.0 version, Didn't face the postgres issue which we faced earlier.
• Run the podman machine init command without image path, as it will installed the latest one.

podman machine init --disk-size 50 --memory 6144 --cpus 2

from podman.

Thanks, @glaydston for sharing the article 🙌🏻

from podman.