evansm7 / vftool Goto Github PK

sudo /usr/local/vftool/vftool/build/vftool -k vmlinuz -d CentOS-7-aarch64-Minimal-2009.iso -m 1024 -a "console=hvc0"
2021-07-21 10:23:20.047 vftool[80046:6672075] vftool (v0.3 10/12/2020) starting
2021-07-21 10:23:20.048 vftool[80046:6672075] +++ kernel at vmlinuz, initrd at (null), cmdline 'console=hvc0', 1 cpus, 1024MB memory
2021-07-21 10:23:20.048 vftool[80046:6672075] +++ fd 3 connected to /dev/ttys000
2021-07-21 10:23:20.048 vftool[80046:6672075] +++ Waiting for connection to: /dev/ttys000
2021-07-21 10:24:31.985 vftool[80046:6672075] +++ Attaching disc CentOS-7-aarch64-Minimal-2009.iso
2021-07-21 10:24:31.985 vftool[80046:6672075] +++ Configuration validated.
2021-07-21 10:24:31.985 vftool[80046:6672075] +++ canStart = 1, vm state 0
2021-07-21 10:24:32.295 vftool[80046:6672782] +++ VM started

I can run ubuntu and debian via vftool just fine, vftool with "-a 'console=hvc0'" results in the kernel boot sequence outputting to the interactive terminal and eventually I get to a login prompt as expected.

Emulating aarch64 from an x86-64 host following this (https://xnand.netlify.app/2019/10/03/armv8-qemu-efi-aarch64.html) works like a charm with this command:

qemu-system-aarch64 -M virt -m 1024 -cpu cortex-a57 -kernel ./Image -initrd ./initramfs-linux.img -drive file=arm64disk.img,format=raw,index=0,media=disk -nographic -no-reboot -append "root=/dev/vda2 rw console=ttyAMA0"

I copied the kernel, initrd, and img to my M1 MacBook Air to boot with vftool with this command:
./vftool -k Image -i initramfs-linux.img -d arm64disk.img -m 8192 -a "root=/dev/vda2 rw console=hvc0" -p 4 -t 0

This results in the following output, with the kernel apparently never loading:
2021-01-04 22:18:54.370 vftool[5142:170426] vftool (v0.3 10/12/2020) starting
2021-01-04 22:18:54.370 vftool[5142:170426] +++ kernel at Image, initrd at initramfs-linux.img, cmdline 'root=/dev/vda2 rw console=hvc0', 4 cpus, 8192MB memory
2021-01-04 22:18:54.380 vftool[5142:170426] +++ Attaching disc arm64disk.img
2021-01-04 22:18:54.380 vftool[5142:170426] +++ Configuration validated.
2021-01-04 22:18:54.380 vftool[5142:170426] +++ canStart = 1, vm state 0
2021-01-04 22:18:54.424 vftool[5142:170427] +++ VM started

If I try to run arch's initrd and img with the ubuntu 5.4 kernel, here's the tail end of my output:

...
Starting version 246.1-1-arch
:: running hook [udev]
:: Triggering uevents...
Waiting 10 seconds for device /dev/vda2 ...
ERROR: device '/dev/vda2' not found. Skipping fsck.
:: mounting '/dev/vda2' on real root
mount: /new_root: special device /dev/vda2 does not exist.
You are now being dropped into an emergency shell.
sh: can't access tty; job control turned off

From the initramfs shell, I can confirm that /dev/vda does not exist, even though mkinitcpio should have been loaded with the "block" driver for virtio disks (again, this runs perfectly under emulation on the x86-64 host)

I've been trying to set up Ubuntu VM on my M1 mac with vftool. However the VM seems to quit immediately after I screen into it:

vftool/build/vftool -k kernel -i initrd -d disk.img -m 4096 -a "console=hvc0"

Then, on another tab, I do screen /dev/ttys002. The screen gets terminated immediately:

❯ screen /dev/ttys002
[screen is terminating]

And the VM exits as well:

❯ ./vftool -k vm/vmlinuz -i vm/initrd -d vm/focal-server-cloudimg-arm64.img -m 6144 -a "console=hvc0"

2022-03-16 15:48:35.392 vftool[11218:150291] vftool (v0.3 10/12/2020) starting
2022-03-16 15:48:35.392 vftool[11218:150291] +++ kernel at vm/vmlinuz, initrd at vm/initrd, cmdline 'console=hvc0', 1 cpus, 6144MB memory
2022-03-16 15:48:35.393 vftool[11218:150291] +++ fd 3 connected to /dev/ttys002
2022-03-16 15:48:35.393 vftool[11218:150291] +++ Waiting for connection to:  /dev/ttys002
2022-03-16 15:48:44.302 vftool[11218:150291] +++ Attaching disc vm/focal-server-cloudimg-arm64.img
2022-03-16 15:48:44.303 vftool[11218:150291] +++ Configuration validated.
2022-03-16 15:48:44.303 vftool[11218:150291] +++ canStart = 1, vm state 0
2022-03-16 15:48:44.449 vftool[11218:150341] +++ VM started
2022-03-16 15:48:45.309 vftool[11218:150291] +++ Done, state = 3

Any ideas what causes this? I tried to follow this tutorial

Hi,

I was trying to create a socket device

In getVMConfig function :-

VZVirtioSocketDeviceConfiguration *socketVirtioConf = [[VZVirtioSocketDeviceConfiguration alloc] init];

NSArray *socketVirtioArray = [NSArray arrayWithObject:socketVirtioConf];
[conf setSocketDevices:socketVirtioArray];

In main after VZVirtualMachine init
VZVirtioSocketDevice *socketDevice = (VZVirtioSocketDevice *)[vm.socketDevices objectAtIndex:0]

I am able to get the socketDevice. But when i try to call [socketDevice setSocketListener:listener forPort:19923]; I get illegal hardware exception. Can you help me out ?

I use the Centos Image file and This is my error :
VM start error: Error Domain=VZErrorDomain Code=1 "The virtual machine failed to start." UserInfo={NSLocalizedFailure=Internal Virtualization error., NSLocalizedFailureReason=The virtual machine failed to start.}

The macOS 12 SDK has new APIs to virtualize macOS, e.g. https://developer.apple.com/documentation/virtualization/vzmacosbootloader?language=objc

Are there any plans to extend vftool to support this use-case?

Just wondering if there's any way to request a particular IP address when starting a VM? Or if not, write a file somewhere containing it so that it can be read in by other things? I'm running PostgreSQL from inside a VM and wanting to connect to it. But I'd rather not have to sign in each time to see what IP address I need to connect to.

I Shawn Daniel McLean need to fix this issue so I can communicate telepathically clearly my interfaces were stolen and I need to get a repo executed immediately

Surprised to see a CLI tool not available via brew. Any plans to publish that way?

could you give us some pointers about how you created your debian install...

the issue I have is there do not appear to be any 'live' images for arm64, just intel/amd

Ive worked out the basics of how to pull a vmlinuz and initrd from an install image.
so I can boot the netinst image... but it doesnt find the network driver

Im guessing this is perhaps related to your comment

An example working commandline is:

    vftool -k ~/vm/debian/Image-5.9 -d ~/vm/debian/arm64_debian.img  -p 2 -m 4096 -a "console=hvc0 root=/dev/vda1"
I've used a plain/defconfig Linux 5.9 build (not gzipped):

    $ file Image-5.9
    Image-5.9: Linux kernel ARM64 boot executable Image, little-endian, 4K pages
Note that Virtualization.framework provides all IO as virtio-pci, including the console (i.e. not a UART). The debian install kernel does not have virtio drivers, unfortunately. I ended up using debootstrap (--foreign) to install to a disc image on a Linux box... but I hear good things about Fedora etc.

in particular Note that Virtualization.framework provides all IO as virtio-pci, including the console (i.e. not a UART). The debian install kernel does not have virtio drivers

could you provide a clue/reference to what you mean by

I ended up using debootstrap (--foreign) to install to a disc image on a Linux box

Id happy create an image on a linux box if thats how I can move forward :)

Ive seen others using the unbuntu cloud image, unfortunately this is causing me issues since I seem not be able to use qemu-user-static on it, I believe because binfmt_misc is not compiled as a kernel module.
I guess I could recompile the kernel...

but really I wanted a debian stretch image anyway, so if I could boot a debian install that be perfect :)

thanks for any pointers

Hello,
I am very interested in trying this.
I am trying ubuntu-20.04.1-live-server-arm64.iso and would like to perform a install. so I also created a DMG. But it does not seems to launch the installer ... do you have an idea what I am doing wrong ?

I use the following parameters
./vftool -k /Users/simonph/Desktop/Ubuntu/UbuntuServer/vmlinuz -i /Users/simonph/Desktop/Ubuntu/UbuntuServer/initrd -d /Users/simonph/Desktop/Ubuntu/VM.dmg -c /Users/simonph/Desktop/Ubuntu/UbuntuServer/ubuntu-20.04.1-live-server-arm64.iso

And here is the console

When I try using a bridge interface, I always end up with the following kernel panic. Something I miss?

Debin 10
Kernel 5.12

[    1.672712] vmbr0: port 1(enp0s1) entered disabled state
[    1.672824] ------------[ cut here ]------------
[    1.672959] kernel BUG at drivers/net/virtio_net.c:1664!
[    1.673029] Internal error: Oops - BUG: 0 [#1] PREEMPT SMP
[    1.673082] Modules linked in: bridge stp llc bonding fuse crct10dif_ce sha3_ce sha3_generic sha512_ce vhost_net vhost tap vhost_iotlb ip_tables x_tables ipv6 dm_mod
[    1.673266] CPU: 5 PID: 421 Comm: ip Tainted: G        W         5.12.12 #1
[    1.673332] pstate: 60400005 (nZCv daif +PAN -UAO -TCO BTYPE=--)
[    1.673436] pc : virtnet_send_command+0x150/0x158
[    1.673487] lr : virtnet_send_command+0x50/0x158
[    1.673551] sp : ffff8000223db160
[    1.673568] x29: ffff8000223db160 x28: 0000000000000000 
[    1.673603] x27: ffff8000223db790 x26: ffff000035430610 
[    1.673667] x25: 0000000000000005 x24: 0000000000000000 
[    1.673704] x23: ffff8000223db248 x22: ffff80001149f948 
[    1.673756] x21: ffff0000386e3400 x20: ffff80001149f948 
[    1.673804] x19: ffff0000347f78c0 x18: 0000000000000010 
[    1.673885] x17: 0000000000000000 x16: 0000000000000000 
[    1.673950] x15: 000067afc46594a2 x14: 00000000000000da 
[    1.674023] x13: 000000000000006b x12: ffff80001149f948 
[    1.674142] x11: ffff8000223db238 x10: ffff8000223db280 
[    1.674312] x9 : ffff0000347f7000 x8 : ffff0000347f70b0 
[    1.674470] x7 : 0000000000000000 x6 : ffff0000347f7000 
[    1.674493] x5 : ffff000039cc3000 x4 : ffff800011596ae8 
[    1.674539] x3 : ffff800011596ca0 x2 : 0000000000000011 
[    1.674632] x1 : 0000000000000011 x0 : 00000005300119ab 
[    1.674686] Call trace:
[    1.674722]  virtnet_send_command+0x150/0x158
[    1.674800]  virtnet_set_guest_offloads+0x58/0xa8
[    1.674849]  virtnet_set_features+0x44/0x70
[    1.674898]  __netdev_update_features+0x1d8/0x6e0
[    1.674987]  netdev_update_features+0x2c/0x68
[    1.675074]  dev_disable_lro+0x24/0x120
[    1.675135]  br_add_if+0x4b8/0x750 [bridge]
[    1.675287]  br_add_slave+0x14/0x20 [bridge]
[    1.675358]  do_set_master+0x90/0xb8
[    1.675415]  do_setlink+0x32c/0xde0
[    1.675489]  __rtnl_newlink+0x3e8/0x798
[    1.675526]  rtnl_newlink+0x4c/0x78
[    1.675567]  rtnetlink_rcv_msg+0x120/0x338
[    1.675613]  netlink_rcv_skb+0x5c/0x120
[    1.675654]  rtnetlink_rcv+0x18/0x28
[    1.675719]  netlink_unicast+0x1b8/0x278
[    1.675748]  netlink_sendmsg+0x1ac/0x3b8
[    1.675770]  ____sys_sendmsg+0x210/0x250
[    1.675787]  ___sys_sendmsg+0x84/0xc8
[    1.675824]  __sys_sendmsg+0x5c/0xa8
[    1.675876]  __arm64_sys_sendmsg+0x24/0x30
[    1.675897]  do_el0_svc+0xe0/0x158
[    1.675927]  el0_svc+0x30/0x98
[    1.675955]  el0_sync_handler+0x90/0xd0
[    1.675978]  el0_sync+0x160/0x180
[    1.676041] Code: f94023f9 a8cc7bfd d50323bf d65f03c0 (d4210000) 
[    1.676099] ---[ end trace ad1cc8642fef743a ]---
[    1.676160] note: ip[421] exited with preempt_count 1
[    1.676298] ------------[ cut here ]------------
[    1.676435] WARNING: CPU: 5 PID: 0 at kernel/rcu/tree.c:637 rcu_eqs_enter.isra.82+0x84/0x90
[    1.676597] Modules linked in: bridge stp llc bonding fuse crct10dif_ce sha3_ce sha3_generic sha512_ce vhost_net vhost tap vhost_iotlb ip_tables x_tables ipv6 dm_mod
[    1.676851] CPU: 5 PID: 0 Comm: swapper/5 Tainted: G      D W         5.12.12 #1
[    1.676902] pstate: 204003c5 (nzCv DAIF +PAN -UAO -TCO BTYPE=--)
[    1.677002] pc : rcu_eqs_enter.isra.82+0x84/0x90
[    1.677042] lr : rcu_idle_enter+0x10/0x20
[    1.677076] sp : ffff800011863f10
[    1.677120] x29: ffff800011863f10 x28: 0000000000000000 
[    1.677207] x27: 0000000000000000 x26: ffff0000340ce000 
[    1.677277] x25: 0000000000000000 x24: ffff80001149fadc 
[    1.677325] x23: ffff8000112d78b8 x22: ffff80001149f948 
[    1.677375] x21: ffff80001149fab8 x20: 0000000000000020 
[    1.677469] x19: ffff80001149f9e0 x18: 0000000000000010 
[    1.677516] x17: 0000000000000000 x16: 0000000000000000 
[    1.677550] x15: 00009435f592f5c0 x14: 0000000000000000 
[    1.677620] x13: 0000000000000000 x12: 0000000000000002 
[    1.677676] x11: 0000000000000000 x10: ffff0000351a5800 
[    1.677705] x9 : 000000000000001f x8 : 0000000000000001 
[    1.677777] x7 : 0000000000000000 x6 : 00000160ac8446e7 
[    1.677839] x5 : 00ffffffffffffff x4 : ffff8000116145f0 
[    1.677894] x3 : 4000000000000002 x2 : 4000000000000000 
[    1.677951] x1 : ffff8000112d96c0 x0 : ffff0000afe916c0 
[    1.677990] Call trace:
[    1.678045]  rcu_eqs_enter.isra.82+0x84/0x90
[    1.678091]  rcu_idle_enter+0x10/0x20
[    1.678135]  default_idle_call+0x20/0x5c
[    1.678181]  do_idle+0x204/0x278
[    1.678230]  cpu_startup_entry+0x24/0x68
[    1.678260]  secondary_start_kernel+0x178/0x188
[    1.678291] ---[ end trace ad1cc8642fef743b ]---

Thank you so much for building this. This has been incredibly helpful.

I'm using this to spin up a Docker VM while I wait for Docker to become compatible with Apple Silicon. My VM seems to panic and shut down after a few minutes of pulling Docker images. The logs from vftool don't indicate any panics from AppleVZ, and screenlog.0 doesn't say anything before it shuts down, but I'm unable to find any instances of vftool in my process list after a few minutes.

Am I doing something wrong?

If it helps, I'm starting it on Ansible but am daemonizing it through nohup and shell redirection. The subshell running vftool forks from PID 1 when I do it this way.

i want to change default ubuntu user passwd
but try to in ubuntu user enter passwd to change new pw and reboot , not work

i added persistent storage for / & home in -d data.img (50G)

I am using MacBook Air M1, I followed Issue 2 to setup linux vm. Finally I found "com.apple.Virtualization.VirtualMachine"
runs on 4 efficient core by default （trace the VirtualMachine process with Xcode Instruments）.
is it possible to let vm run on 4 big cores?

Is there a list of what kernel modules are required for an image to boot? I'm trying this on an Intel Mac (Big Sur).

I've been successful in getting an Ubuntu VM to load (using the Ubuntu cloud images and following the instructions from @droidix #2 (comment)). I've also managed to get the cloud image of Debian 10 to work, by adding certain virtio modules to the initrd image, but it's still strange...

Basically, the Debian kernel works with the following modules added:

• virtio_pci
• virtio_blk
• virtio_net
• virtio_console

However -- it doesn't work exactly like this. When loading the kernel with the following options -a "root=/dev/vda console=hvc0", the VM goes right into the initramfs image. Adding virtio_console got me this far.

Steps (in either a chroot from the initramfs image or using the Ubuntu kernel/initrd to edit the Debian initrd)

(initramfs) mkdir /mnt
(initramfs) mount -t ext4 /dev/vda /mnt
root@(none):/# echo "virtio_console" >> /etc/initramfs-tools/modules 
root@(none):/# update-initramfs -u

I then copied the file /boot/initrd.img-4.19.0-13-amd64 image out of the disk image to use with vftool.

The strangeness is that I can get a full VM to boot when I add the debug option to the kernel command. Starting the vm with -a "root=/dev/vda console=hvc0 debug" successfully boots the Debian VM. It seems like there is a timing issue where the storage device isn't found unless the kernel is in debug mode, but I don't have any guess as to where it is.

I'd like to be able to use as stock of a Debian kernel as possible, so does anyone know which kernel modules or arguments I might need to try to get this to work stably?

Here is the full command line that works:

vftool -k vmlinuz-4.19.0-13-amd64 -i newinitrd -d debian_10_cloud.img -t 1 -m 2048 -a "root=/dev/vda console=hvc0 debug"      

This is the command that doesn't:

vftool -k vmlinuz-4.19.0-13-amd64 -i newinitrd -d debian_10_cloud.img -t 1 -m 2048 -a "root=/dev/vda console=hvc0"      

I have the same results if I'm using the full disk with partitions (using root=/dev/vda1) and only the ext4 partition (using root=/dev/vda).

Hello, I create a ubuntu vm use vftool, and I find that the space of "/" is 985M, how to increase it? Thanks.

A few issues here out the box from an xcodebuild I get permissions denied, I then have to CD to ~/vftool/build/Release then I am able to run the following command: ./vftool -k vmlinuz -i initrd -d focal-desktop-arm64.iso -m 4096 -a "console=hvc0" without getting permission denied.

Yet when the VM starts to load from the disk image: focal-desktop-arm64.iso I get the following error:

VM start error: Error Domain=VZErrorDomain Code=2 "The boot loader is invalid." UserInfo={NSLocalizedFailure=Invalid virtual machine configuration., NSLocalizedFailureReason=The boot loader is invalid., NSUnderlyingError=0x13e6187f0 {Error Domain=VZErrorDomain Code=2 "Failed to open the kernel file." UserInfo={NSLocalizedFailure=Invalid virtual machine configuration., NSLocalizedFailureReason=Failed to open the kernel file., NSUnderlyingError=0x13e704370 {Error Domain=NSPOSIXErrorDomain Code=2 "No such file or directory"}}}}

OS Is Big Sur using Xcode 12

I very want to try it, but I'm not a apple developer, can you publish a release version?

Thanks for this brilliant tool which is exactly what I wanted. I followed the helpful guide #2 (comment)
to get a Ubuntu VM which is very fast in single user mode (no -p option), but as soon as I enable more than one core, performance is very very slow.

Is this a known issue?

As you all know,the best application runtime environment on ARM64 is Android.But I don't know whether the Android Linux will fit into the macOS Virtualization framework or not,if so,the graphics acceleration API access will also be good

Thanks for making the tool and instructions are pretty clear.

I've had successful launching of vm using the command line. Though, it is possible to connect to the vm, I see that for my activities it says I do not have enough disk space. my question is: would it be possible to add extra disk.volume to the vm? or just bumping up the memory sufficient?