Conference Schedule - BSDCan 2020

Booting a Linux kernel under bhyve on ARMv7

Date: 2020-06-06
Time: 13:30 - 14:15
Room: Stream 3
Level: Intermediate

ARM processors are more energy efficient when compared to their older and more powerful x86 counterparts. As such, more complex systems (e.g., servers) would greatly benefit from using them should they become powerful enough to be able to handle complex tasks. One such task, that is an essential tool for system administrators, is the ability to run virtual machines in order to provide secure and isolated environments for certain applications. With ARM-powered servers being under development for years already, anticipating the needs of system administrators and adding relevant features to the operating system may prove critical to increase the user base.

Linux is by far the most successful free operating system, so any virtualization mechanism will need to be able to run Linux as a guest before it may be considered an option. Consequently, bhyve, FreeBSD's virtualization manager will need to support it on ARM systems. This presentation presents our testing of the boot process for a Linux guest under bhyve on ARMv7, some issues that were uncovered while testing and further work that needs to be done in order to get a fully functional hypervisor on ARM-based systems.

To create a functional virtualization environment that is able to run any guest operating system under FreeBSD, bhyve, the virtual machine manager, will have to support the VirtIO devices. VirtIO devices are a paravirtualization mechanism for efficient communication between guest and host, created by the OASIS Committee. The virtualized devices are supported by all major operating systems (i.e., special device drivers are implemented) for low overhead data transfers. An emulated data bus is created by the host and the VirtIO devices are added to this bus. From the guest's point of view, the devices should be accessible using regular bus access mechanisms (e.g., the x86 implementation uses an emulated PCI bus, where device iteration and polling can be used), and can be identified by the drivers using certain magic values.

As opposed to the x86 implementation that relies on PCI device emulation, devices connected to an ARMv7 system on a chip can only use MMIO addressing and a implements a different interrupt mechanism. Each device must have an individual memory location and interrupt number assigned for them. Operating systems for ARM systems use a Flattened Device Tree to specify the board's device structure in a unique pseudo-hierarchical manner. The hierarchical structure enables device drivers to iterate over the installed devices and identify compatible devices.

Due to a lack of boards that could be used to test the device functionality, we began testing on the ARM Fast Models system emulator. The test scenario uses a FreeBSD host image that incorporates a barebone root filesystem with only the programs required to start the guest operating system and communicate with it, and the image of the guest operating system. A further simplification is that the guest operating system also contains the root filesystem in its image.

A FreeBSD guest can be easily created within these limitations, since the build system provides options to include both the root filesystem and a device tree blob, which are easily picked up by the operating once it boots. However, the build system for the Linux operating system does not provide such mechanisms out of the box. Instead, various other tools were needed to create a complete image that could be used as a guest operating system that boots completely. The presentation provides insight into how a Linux guest needed to be created to make it compatible with the bhyve hypervisor, what implementation errors were discovered while booting the Linux guest, and what the next steps should be in implementing bhyve as a hypervisor for ARM-based systems.



Darius Mihai