Introduction
In this tutorial, you will learn how to compile both qemu and libvirt from scratch without disturbing the existing qemu and libvirt daemons running on Linux® on IBM® Power Systems™ servers.
In this tutorial, you can learn to:
- Clone the upstream libvirt and qemu code
- Configure, build, and install upstream qemu code
- Configure, build, and install upstream libvirt code
- Run the libvirt/qemu instance
Prerequisites
Basic knowledge of Linux and a working Linux system is a prerequisite to practice the commands and steps covered in this tutorial.
Use the following script to install the required packages on the Linux on Power system before configuring and building the libvirt and qemu code.
sudo dnf install -y automake gcc make glibc glibc-utils glib2-devel zlib-devel pixman-devel flex bison \
numad numactl-devel numactl-libs numactl \
libtool gnutls-utils gnutls-devel libnl3-devel libxml2-devel \
libtirpc-devel python3-docutils device-mapper-devel libpciaccess-devel \
rpcbind readline-devel rpcgen yajl-devel libxslt-devel bzip2
Estimated time
It would take approximately 1 hour to complete the setup for configuring and running libvirt and qemu instances.
Steps
Install Git and clone both upstream libvirt and qemu repos.
sudo dnf install -y git git clone https://github.com/qemu/qemu.git git clone https://github.com/libvirt/libvirt.gitConfigure and build the qemu code and then install the qemu code. The
make installcommand creates the binary files in the /usr/local directory.cd qemu mkdir -p build cd build # if you want the x86 target too you can add "x86-softmmu" in the target list ../configure --enable-trace-backend=simple --enable-debug --target-list=ppc64-softmmu --prefix=/usr/local make -j sudo make installConfigure and build libvirt and then install the libvirt code.
libvirt_buildcontains the log and configuration files for the daemons and the qemu driver.cd $HOME mkdir -p libvirt_build cd libvirt(—- for libvirt 6.6.0 and older —-)
mkdir build cd build ../autogen.sh --prefix=$HOME/libvirt_build make -j # make install is required only once to generate the config and log file structure make install(—- for libvirt 6.7.0 and later —-)
$ meson build --prefix=$HOME/libvirt_build $ ninja -C build # ninja -C build install is required only once to generate the config and log file structure $ sudo ninja -C build installAfter running step 1,2 and 3 you will notice that the following qemu binary files are compiled and created in the /usr/local/bin directory.
$ ls /usr/local/bin ivshmem-client ivshmem-server qemu-edid qemu-ga qemu-img qemu-io qemu-nbd qemu-pr-helper qemu-storage-daemon qemu-system-ppc64Also, the libvirt binary files are in $HOME/libvirt/build.
These binary files are in fact inside another directory, but there is a helper script called ‘run’ in this directory that you can use to run these binary files. This helper script sets up the environment variables to run the libvirt utilities.
Run the following commands from the $HOME/libvirt/build directory to start the libvirt daemon (libvirtd).
sudo ./run src/virtlockd &(must be running in the background)sudo ./run src/virtlogd &(must be running in the background)sudo ./run src/libvirtd(the libvirtd daemon I run in the foreground to see logs)The configuration files for these compiled daemons are located in $HOME/libvirt_build/etc/libvirt.
The ./run script can be used to run the compiled version of the tools.
After starting the libvirtd daemon, use the
virshcommands to work with the virtual machines (VMs).This is how I work with the compiled
virsh:build]$ pwd /home/sthoufee/libvirt/build build]$ build]$ sudo ./run tools/virsh destroy apic_test Domain apic_test destroyed build]$ sudo ./run tools/virsh list --all Id Name State ---------------------------- - apic_test shut offNotice that the compiled
virshcommand is used to destroy the VM and if you view the VM list, you can see the status asshut offconfirming that the virtual machine is destroyed.
Additional information
You can view the configuration summary by running libvirt ./autogen.sh. In the output you can see if the qemu driver is enabled.
-----
configure: Configuration summary
configure: =====================
configure:
configure: Drivers
configure:
configure: QEMU: yes <----------
(...)
-----
If qemu is not enabled, you’re probably missing some package in your setup. libvirt will actually run without the qemu driver (because libvirt is not dependent on qemu) but then you’ll be unable to run qemu VMs without the driver. The /etc/libvirt/qemu.conf file will be missing too.
Note that you must use the compiled virsh to handle the compiled libvirt. Using virsh from the system installation will work with the libvirt from the system.
Here is an example output of the virt process running in my system.
[build]$ ps axf | grep virt
36770 pts/1 S 0:00 | \_ sudo ./run src/virtlockd
36778 pts/1 S 0:00 | | \_ /home/sthoufee/libvirt/build/src/.libs/lt-virtlockd
36814 pts/1 S 0:00 | \_ sudo ./run src/virtlogd
36816 pts/1 S 0:00 | | \_ /home/sthoufee/libvirt/build/src/.libs/lt-virtlogd
76055 pts/1 S+ 0:00 | \_ sudo ./run src/libvirtd
76063 pts/1 Sl+ 0:00 | \_ /home/sthoufee/libvirt/build/src/.libs/lt-libvirtd
135149 pts/2 S+ 0:00 \_ grep --color=auto virt
192443 ? Ss 0:00 /usr/sbin/virtlockd
192452 ? Ss 0:00 /usr/sbin/virtlogd
192462 ? Ssl 0:02 /usr/sbin/libvirtd
192632 ? S 0:03 /usr/sbin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro --dhcp-script=/usr/libexec/libvirt_leaseshelper
192633 ? S 0:00 \_ /usr/sbin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro --dhcp-script=/usr/libexec/libvirt_leaseshelper
In this process, everything being run from ‘/usr/sbin’ is from the system based libvirt OS installation. The virt process that been run as sudo is from custom libvirt. This indicates that you can have system-based libvirtd daemon and custom libvirtd daemon both running at the same time.
The following command lists the virtual machine running on a custom libvirt instance.
[build]$ sudo ./run tools/virsh list --all (handles custom libvirt)
Id Name State
----------------------------
- apic_test shut off
[build]$
[build]$
The following command lists the VMs running on a system-based libvirt instance.
[build]$ sudo virsh list --all (handles system libvirt)
Id Name State
----------------------------------------------------
- apic_test shut off
- memhotplug shut off
- vcpupintest shut off
Summary
You will have local qemu binary files in the /usr/local/bin directory and a local libvirt installation at $HOME/libvirt/build directory (with configuration files at $HOME/libvirt_build/etc/libvirt), and these binary files should co-exist with the existing qemu and libvirt installations in the system. With the procedure mentioned in this tutorial, you should be able to have two libvirt daemons (libvirtd) running in the system (one from the system and one compiled from source) without one messing with the other.