Introduction

In this tutorial, you will learn how to compile both qemu and libvirt from scratch without disturbing the existing qemu and libvirt daemons running on Linux® on IBM® Power Systems™ servers.

In this tutorial, you can learn to:

• Clone the upstream libvirt and qemu code
• Configure, build, and install upstream qemu code
• Configure, build, and install upstream libvirt code
• Run the libvirt/qemu instance

Prerequisites

Basic knowledge of Linux and a working Linux system is a prerequisite to practice the commands and steps covered in this tutorial.

Use the following script to install the required packages on the Linux on Power system before configuring and building the libvirt and qemu code.

 sudo dnf install -y automake gcc  make glibc glibc-utils glib2-devel zlib-devel pixman-devel flex bison \
 numad numactl-devel numactl-libs numactl \
 libtool gnutls-utils  gnutls-devel  libnl3-devel libxml2-devel \
 libtirpc-devel python3-docutils device-mapper-devel libpciaccess-devel \
 rpcbind  readline-devel rpcgen yajl-devel libxslt-devel bzip2

Estimated time

It would take approximately 1 hour to complete the setup for configuring and running libvirt and qemu instances.

Steps

1. Install Git and clone both upstream libvirt and qemu repos.

   sudo dnf install -y git
   git clone https://github.com/qemu/qemu.git
   git clone https://github.com/libvirt/libvirt.git
   

   Show more

2. Configure and build the qemu code and then install the qemu code. The make install command creates the binary files in the /usr/local directory.

     cd qemu
     mkdir -p build
     cd build
     # if you want the x86 target too you can add "x86-softmmu" in the target list
     ../configure --enable-trace-backend=simple --enable-debug --target-list=ppc64-softmmu --prefix=/usr/local
     make -j
     sudo make install
   

   Show more

3. Configure and build libvirt and then install the libvirt code.

   libvirt_build contains the log and configuration files for the daemons and the qemu driver.

   cd $HOME
   mkdir -p libvirt_build
   cd libvirt
   

   Show more

   (—- for libvirt 6.6.0 and older —-)

   mkdir build
   cd build
   ../autogen.sh --prefix=$HOME/libvirt_build
   make -j
   # make install is required only once to generate the config and log file structure
   make install
   

   Show more

   (—- for libvirt 6.7.0 and later —-)

   $ meson build --prefix=$HOME/libvirt_build
   $ ninja -C build
   # ninja -C build install is required only once to generate the config and log file structure
   $ sudo ninja -C build install
   

   Show more

4. After running step 1,2 and 3 you will notice that the following qemu binary files are compiled and created in the /usr/local/bin directory.

   $ ls  /usr/local/bin
   ivshmem-client  ivshmem-server  qemu-edid  qemu-ga  qemu-img  qemu-io  qemu-nbd  qemu-pr-helper  qemu-storage-daemon  qemu-system-ppc64
   

   Show more

   Also, the libvirt binary files are in $HOME/libvirt/build.

   These binary files are in fact inside another directory, but there is a helper script called ‘run’ in this directory that you can use to run these binary files. This helper script sets up the environment variables to run the libvirt utilities.

5. Run the following commands from the $HOME/libvirt/build directory to start the libvirt daemon (libvirtd).

   sudo ./run src/virtlockd & (must be running in the background) sudo ./run src/virtlogd & (must be running in the background) sudo ./run src/libvirtd (the libvirtd daemon I run in the foreground to see logs)

   The configuration files for these compiled daemons are located in $HOME/libvirt_build/etc/libvirt.

   The ./run script can be used to run the compiled version of the tools.

6. After starting the libvirtd daemon, use the virsh commands to work with the virtual machines (VMs).

   This is how I work with the compiled virsh:

   build]$ pwd
   /home/sthoufee/libvirt/build
   build]$
   build]$ sudo ./run tools/virsh destroy  apic_test
   Domain apic_test destroyed
   
   build]$ sudo ./run tools/virsh list --all
   
   Id   Name        State
   ----------------------------
   -    apic_test   shut off
   

   Show more

   Notice that the compiled virsh command is used to destroy the VM and if you view the VM list, you can see the status as shut off confirming that the virtual machine is destroyed.

Additional information

You can view the configuration summary by running libvirt ./autogen.sh. In the output you can see if the qemu driver is enabled.

   -----
   configure: Configuration summary
   configure: =====================
   configure:
   configure: Drivers
   configure:
   configure:                 QEMU: yes <----------
   (...)
   -----

If qemu is not enabled, you’re probably missing some package in your setup. libvirt will actually run without the qemu driver (because libvirt is not dependent on qemu) but then you’ll be unable to run qemu VMs without the driver. The /etc/libvirt/qemu.conf file will be missing too.

Note that you must use the compiled virsh to handle the compiled libvirt. Using virsh from the system installation will work with the libvirt from the system.

Here is an example output of the virt process running in my system.

 [build]$ ps axf | grep virt
 36770 pts/1    S      0:00  |   \_ sudo ./run src/virtlockd
 36778 pts/1    S      0:00  |   |   \_ /home/sthoufee/libvirt/build/src/.libs/lt-virtlockd
 36814 pts/1    S      0:00  |   \_ sudo ./run src/virtlogd
 36816 pts/1    S      0:00  |   |   \_ /home/sthoufee/libvirt/build/src/.libs/lt-virtlogd
 76055 pts/1    S+     0:00  |   \_ sudo ./run src/libvirtd
 76063 pts/1    Sl+    0:00  |       \_ /home/sthoufee/libvirt/build/src/.libs/lt-libvirtd
 135149 pts/2    S+     0:00      \_ grep --color=auto virt
 192443 ?        Ss     0:00 /usr/sbin/virtlockd
 192452 ?        Ss     0:00 /usr/sbin/virtlogd
 192462 ?        Ssl    0:02 /usr/sbin/libvirtd
 192632 ?        S      0:03 /usr/sbin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro --dhcp-script=/usr/libexec/libvirt_leaseshelper
 192633 ?        S      0:00  \_ /usr/sbin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro --dhcp-script=/usr/libexec/libvirt_leaseshelper

In this process, everything being run from ‘/usr/sbin’ is from the system based libvirt OS installation. The virt process that been run as sudo is from custom libvirt. This indicates that you can have system-based libvirtd daemon and custom libvirtd daemon both running at the same time.

The following command lists the virtual machine running on a custom libvirt instance.

[build]$ sudo ./run tools/virsh list --all       (handles custom libvirt)

 Id   Name        State
 ----------------------------
 -    apic_test   shut off

[build]$
[build]$

The following command lists the VMs running on a system-based libvirt instance.

[build]$ sudo virsh list --all      (handles system libvirt)

 Id    Name                           State
 ----------------------------------------------------
 -     apic_test                      shut off
 -     memhotplug                     shut off
 -     vcpupintest                    shut off

Summary

You will have local qemu binary files in the /usr/local/bin directory and a local libvirt installation at $HOME/libvirt/build directory (with configuration files at $HOME/libvirt_build/etc/libvirt), and these binary files should co-exist with the existing qemu and libvirt installations in the system. With the procedure mentioned in this tutorial, you should be able to have two libvirt daemons (libvirtd) running in the system (one from the system and one compiled from source) without one messing with the other.