3: Xen: Hypervisor-based virtualization

To make things more effective than the userspace model (mentioned in part 2) we can use a Hypervisor. The Hypervisor is a small piece of software which controls the hardware - it assigns memory, CPU cycles, PCI hardware. This hypervisor replaces the kernel of a traditional, "real" hardware system:


The hypervisor does not include - however - device drivers - it only controls I/O-spaces, Interrupts, CPU cycles, RAM usage.
On top of this hypervisor there are virtual machines named "Domains". There are two different types of Domains:

  • Domain 0 (short Dom0): This is a virtual machine which is able to control the hypervisor
  • Domain U (short DomU): This is a normal virtual machine running a guest system

The Dom0 and normal DomU's need special guest operating systems because they are so-called paravirtualized domains. The guest operating system does not access on hardware functionalities in low levels of a typical kernel anymore (Ring 0) but it has a standardized API to communicate with the Hypervisor (XENBUS and Xen Virtual Devices).
The Dom0 is somewhat special, because itself it uses "passthru"-drivers to access devices under the hypervisor directly (in OpenSolaris the driver is named xvm_psr). Whatever domain "gets" underlying devices directly prohibits other domains from seeing them. Normally Domain-0 has all devices for passthru and the DomUs do not have any. But this can be changed to allow a DomU to use a Fiberchannel card (as an example) exclusively.

If not defined otherwise - all hardware access (normally storage and network) is done through the Dom0 operating system. That is the reason why Dom0 needs passthru hardware access through the hypervisor.

This behaviour is described by the thin black arrows in the graphic above.

Paravirtualized domains are very efficient as no CPU cycle gets burned to emulate a hardware for these domains. The virtual Xen devices seen by the DomU-virtual machines are handled by the Dom0-system.
Another advantage of this so-called PV domains is the ability to switch virtual CPUs off and on while the guest system is running. Your guest machine needs 3 full CPUs instead of two for an hour to compute something important? Just define the number of "virtual CPUs" to 3 for this domain - the Xen based operating system kernel of the guest will write that an additional CPU has been turned on. After the big calculation you may even set it to 1 - to give more CPU cycles to other domains for example.
And - these paravirtualized domains are not limited to x86 systems. Any Sun SPARC able to run Solaris Express (SPARC) can run the Xen Hypervisor and SPARC paravirtualized domains. Hardware virtualization is not available though.

The main disadvantage of paravirtualized domains is the fact that one of the most  important server operating system does not exist in a Xen-form: Windows.

To be able to run Windows as a DomU you need a hardware virtualized machine. Xen offers that through HVM. You need processors with a hardware virtualization feature called svm internally and AMD-V or Intel VT officially.
The Xen Hypervisor then uses a Bochs BIOS to emulate a real PC and generates Interrupts, PCI busses and all you need to run Windows (and other operating systems).

Take this as an example:


On this graphic you see the "virtual BIOS" created by the Hypervisor to run Windows 2003 and Windows 2008. And you'll notice the big lilac arrows pointing to "qemu-dm" processes running on Domain 0:

qm-demu processes emulate standard hardware like an RTL 8139 network card or an ATA hard disk. They are consuming CPU cycles on Domain 0 to achieve this. This is not a terrific performant solution and should be avoided. To achieve this, you have to install Xen drivers in your guest operating system running in a HVM DomU. If they are existent and installed, the guest operating system will use the same XENBUS and the same virtual Xen devices as paravirtualized domains do - so qemu-dm is not gobbling up CPU time any more.

In this screenshot (click on it to get it in original size) you see a console based paravirtualized OpenSUSE Linux which is accessible with a simple "virsh console"-command from the Domain0-System (OpenSolaris in this case):


For graphical-based guest systems like Microsoft Windows, screen output is accessed via a well-known protocol named VNC (Virtual Network Computing), clients are available for most operating systems. Here is an output of a Windows 2003 Enterprise Server running as Xen HVM DomU (click on the picture to enlarge it):


The "trick" of using Domain-0 as storage- and network aggregator allows (in the case of OpenSolaris as Dom0) to use ZFS as storage pooling system - which makes storage tasks easy.
Just define a ZFS block volume and use that as virtual disk for your DomU. Easy. Fast. And you can make snapshots (to rollback after a broken windows update...).

The main drawback of hypervisor-based virtualization:
It's not well suited for desktop systems. You won't get any graphics acceleration (access via VNC, so it is really just to allow to use the graphical console for system tasks, not more) and it is quite complicated to set up the first time.

It is a solution which is named "Sun xVM" at Sun, and the Xen drivers are still not in a very mature state. The 64bit-Versions for Windows 2008 R2 are still crashing the virtual machine at boot.

For these tasks another hypervisor-based solution named VMWare ESX is much more mature, widespread and (to be honest) is regarded as the most reliable system to virtualize windows hosts. Part 4 of this text deals with ESX.

To (para)virtualize Linux servers however it is (for me) a perfect solution: Nearly no overhead, fast installs (all Linux distributions have a Xen based kernel included, even Suse and Redhat Enterprise Servers).

I am using Sun xVM (Xen 3.4.2) also for my test Active Directory (Windows 2003) in 32bit, because for this combination there are working performant Xen drivers available:


0 TrackBacks

Listed below are links to blogs that reference this entry: 3: Xen: Hypervisor-based virtualization.

TrackBack URL for this entry: http://southbrain.com/mt/mt-tb.cgi/97


Thanks for the heads up on the 2008 Server 64 bit drivers. Nice post, great screen shots. Thanks!


I believe there is now only 1 QEMU for all HVMs in a Xen system. Is that not true?


No :)

pascal@seattle:~$ ps axww|grep qemu
2962 ? S 88:56 /usr/lib/xen/bin/qemu-dm -d 3 -domain-name win2008r2 -videoram 4 -k en-us -vnc
8502 ? S 94:00 /usr/lib/xen/bin/qemu-dm -d 7 -domain-name win_server_2003 -videoram 4 -k en-us

December 2015

Sun Mon Tue Wed Thu Fri Sat
    1 2 3 4 5
6 7 8 9 10 11 12
13 14 15 16 17 18 19
20 21 22 23 24 25 26
27 28 29 30 31    


This blog is owned by:

Pascal Gienger
J├Ągerstrasse 77
8406 Winterthur

Google+: Profile
YouTube Channel: pascalgienger