The Zephyr-based virtual machine (ZVM) is a piece of virtualization software developed by Prof. Guoqi Xie from the Key Laboratory for Embedded and Network Computing of Hunan Province, Hunan University. This software is developed based on Zephyr, a real-time operating system (RTOS). It can enhance both Linux and Zephyr as two separate guest OSs simultaneously, making it ideal for hybrid kernel deployment on a single hardware platform. In February 2023, the Laboratory officially open sourced it in the openEuler community.

1. Background ​

Embedded real-time virtualization is a technology that enables multiple OSs to run on a single hardware platform while maintaining system determinism and time-critical performance. This technology is advantageous for embedded system development, offering benefits such as hardware integration and system isolation, flexibility, reliability, security, and scalability. It can also be applied to advanced applications such as smart cars, computer numerical control (CNC), and 5G devices.

Despite its potential benefits, creating embedded real-time virtualization software presents several issues that need to be addressed. The foremost challenge is guaranteeing isolation and security between various guest OSs, particularly when their levels of importance or reliability vary. Another hurdle is distributing or assigning I/Os between different guest OSs, which may require device simulation or pass-through mechanisms. Besides, how to ensure the RTOS functioning as the guest OS could have low latency and high throughput poses another challenge.

To address these issues, it's necessary to take certain measures such as mandatory isolation and security, efficient interrupt processing, flexible I/O device management, and appropriate hardware. Based on the hardware virtualization and virtual host expansion, the Laboratory developed the ZVM, which handles OS isolation, device allocation, and interrupt processing, while offering superior system security and real-time performance.

2. Technology Overview ​

Secure isolation, device management, and system performance improvement are the three major aspects on which ZVM focuses.

(1) Secure isolation: The virtualization technology allows co-existence of applications with varying levels of privileges by ensuring isolation and security between guest OSs. A virtual address space and a virtual device are allocated to each guest OS, isolating VMs from each other to enhance the overall system security.

(2) Device management: A management program supporting device simulation and pass-through mechanisms is employed to ensure efficient sharing or allocation of I/O devices between guest OSs. I/O devices that need to be exclusively occupied by the interrupt controller are allocated in full virtualization mode, whereas non-exclusive devices such as UART are allocated in device pass-through mode.

(3) System performance improvement: ARM64 hardware-assisted virtualization minimizes context overheads, and hardware-based two-stage address translation reduces memory overheads. What's more, hardware-based interrupt injection reduces context overheads and interrupt delays.

3. System Architecture ​

The following figure shows the overall architecture of the ZVM. Virtualization modules are added to the Zephyr to virtualize CPUs, memory, interrupts, timer, and I/Os. The ZVM accommodates two types of guests OSs: Linux and Zephyr.

**(1) **CPU virtualization: The main function of the CPU virtualization module is to create an independent isolation context for each guest OS's vCPUs. Each vCPU operates as a thread and is scheduled by the ZVM. To improve vCPU performance, ARM64 implements VHE for the ZVM. The VHE redirects ARM registers, and allows host OSs to be ported to the EL2 privilege level without the need for modifying the Zephyr RTOS kernel code. This simplifies the system architecture and improves overall system performance.

(2) Memory virtualization: The memory virtualization module's primary objective is to isolate memory addresses between guest OSs. The system protects physical memory by isolating the memory space of each guest OS and monitoring their access to the memory. To facilitate this function, ARM64 incorporates a two-stage address lookup policy. At the first stage, the virtual address of the guest OS is converted into a physical address; and at the second stage, the physical address of the guest OS is converted into a physical address of the host OS. ARM64 provides independent hardware for the second-stage translation to improve the address translation performance.

(3) Interrupt virtualization: The interrupt virtualization module leverages the Arm general interrupt controller (GIC) to implement virtual interrupt configurations. The interrupts of the guest OS are routed to the ZVM, and the ZVM allocates them to different vCPUs. Virtual interrupts are injected through the Virtual CPU API or List Register of the GIC.

(4) Timer virtualization: The timer virtualization module defines a set of virtual timer registers for each CPU. The registers operate independently and throw interrupts after a specified period of time. These interrupts are transmitted to the guest OS from the host OS. When switching between guest OSs, the virtual timer calculates their respective running times and compensates for any time lost during guest OS exits, thus providing the timer service.

(5) Device virtualization: For device virtualization, the ZVM uses memory-mapped I/O (MMIO) of Arm to map the device address to the virtual address memory to create a virtual device space and allow the guest OS to access the device address. Specifically, the ZVM creates a virtual MMIO device, and allocates it to the specified guest OS during the creation of the guest OS, to achieve I/O virtualization. In addition, for non-exclusive devices, the ZVM implements device access in device passthrough mode.

4. Future Plan ​

The ZVM is maintained by Sig-Zephyr. The SIG plans to launch the first base version that supports the latest Zephyr LTS in May 2023, incorporate the ZVM with openEuler Embedded in July, and improve other advanced functions, such as I/O device framework, dynamic VM resource management, and real-time systems.

Project details:

https://gitee.com/openeuler/zvm

About the Team ​

The Key Laboratory for Embedded and Network Computing of Hunan Province has been engaged in research on embedded computing and systems for a long time. Prof. Guoqi Xie is the director of the Laboratory, and Prof. Wanli Chang is the chief scientist. The Laboratory is committed to fostering a conducive environment for academic research, elevating its status to one of the leading labs globally, and nurturing talent for knowledge innovation, scholarly exchange, and societal development.

Through industry-academia-research collaboration, the Laboratory, with a focus on the world's cutting-edge scientific research achievements, has developed a series of products, such as embedded real-time virtualization software, vehicle-mounted network comprehensive verification platform, and automobile software toolchain. According to CSRrankings, over the past five years (2018-2022), the Laboratory has secured a global ranking of 17 in terms of the number of papers published at top-level academic conferences pertaining to embedded and real-time systems. Additionally, the Laboratory holds the top position in China for the same category. The Laboratory boasts a distinguished team of academic leaders with immense influence on both Chinese and global platforms, alongside a cohort of young and promising technical experts. Notably, Prof. Wanli Chang holds the esteemed position of the Secretary-General of the Special Interest Group on Embedded Systems (SIGBED), Association for Computing Machinery (ACM). With 36 researchers possessing doctoral degrees, including 16 professors and 4 national talent project awardees, the Laboratory is well-positioned to lead the charge in cutting-edge research.