IBM Developer Day | Bengaluru | March 14th Register now
By Jeffrey Osier-Mixon | Updated August 17, 2018 - Published August 28, 2012
The Yocto Project is an open source collaboration project that provides templates, tools, and methods supporting custom Linux-based systems for embedded products, regardless of the hardware architecture. For those of you who are wondering about the name, the term yocto is the smallest SI unit. As a prefix, yocto indicates 10^-24.
This article provides a step-by-step walk through on how to use the Yocto Project’s industry-standard open source tools to create a customized Linux operating system for an embedded device and to boot the operating system in a virtual machine using QEMU. The Yocto Project, a Linux Foundation-sponsored open source project funded by major hardware companies and operating systems vendors, provides industry-class tools, methods, and metadata for building Linux systems.
Two major components of the Yocto Project are maintained in conjunction with the OpenEmbedded project: BitBake, the build engine, and OpenEmbedded-Core, the core set of recipes used to run the build process. All of the project’s components are described in the next section.
As a collaboration project, sometimes called an “umbrella” project, the Yocto Project incorporates many different disparate pieces of the development process. These pieces are referred to as projects within the overall Yocto Project, and they include build tools, build instruction metadata called recipes, libraries, utilities, and graphical user interfaces (GUIs).
Poky is a reference build system for the Yocto Project. It includes BitBake, OpenEmbedded-Core, a board support package (BSP), and any other packages or layers incorporated into the build. The name Poky also refers to the default Linux distribution resulting from using the reference build system, which can be extremely minimal (core-image-minimal) or a full Linux system with a GUI (core-image-sato).
You can think of the Poky build system as a reference system for the entire project—a working example of the process in action. When you download the Yocto Project, you actually download an instance of those tools, utilities, libraries, toolchain, and metadata that you can use to build the default system, as described here. That reference system and the reference distribution it creates are both named Poky. You can also use this as a starting point to create your own distribution, which of course you can name anything you like.
One item that all build systems require is a toolchain: a compiler, assembler, linker, and other binary utilities necessary for creating binary executable files for a given architecture. Poky uses the GNU Compiler Collection (GCC), but you can specify other toolchains as well. Poky uses a technique known as cross-compilation: using a toolchain on one architecture to build binary executable files for a different architecture (for example, building an ARM distribution on an x86-based system). Developers often use cross-compilation in embedded systems development to take advantage of the host system’s higher performance.
The metadata set is arranged in layers, such that each layer can provide separate functionality to the layers beneath it. The base layer is OpenEmbedded-Core, or oe-core, which provides recipes, classes, and associated functions that are common and necessary for all builds. You can then customize builds by adding new layers on top of oe-core.
OpenEmbedded-Core is co-maintained by the Yocto Project and the OpenEmbedded project. One layer that separates the Yocto Project from OpenEmbedded is the meta-yocto layer, which provides the Poky distribution configuration and a core set of reference BSPs.
The OpenEmbedded project itself is a separate open source project with (largely) interchangeable recipes and similar goals to the Yocto Project, but different governance and scope.
A BSP contains the essential packages and drivers necessary for building Linux for a specific board or architecture. These are often maintained by the hardware manufacturers who make the boards. BSPs are the interface between the Linux operating system and the hardware that runs it. Note that it is also possible to create BSPs for virtual machines.
BitBake is a build engine. It reads recipes and follows them by fetching packages, building them, and incorporating the results into bootable images. BitBake is co-maintained by the Yocto Project and the OpenEmbedded project.
In an effort to make embedded Linux development easier, the Yocto Project provides a few different methods for working graphically. A relatively new addition to the project is called Hob, which provides a graphical front end to BitBake and the build process. Both are under continual development, complete with community user studies.
Complying with open source licenses is an extremely important part of any Linux development effort. One goal of the Yocto Project is to make compliance as easy as possible. It is quite easy to use the Yocto Project tools to create manifests and even to build entire source repositories, as well as filtering your build process to exclude packages that use specific licenses. The project is working with the Linux Foundation on its Open Compliance Program in relation to the Software Package Data Exchange® (SPDX™) specification.
Embedded GLIBC (EGLIBC) is a variant of the GNU C Library (GLIBC) that is designed to work well on embedded systems. EGLIBC’s goals include reduced footprint, configurable components, and better support for cross-compilation and cross-testing. EGLIBC is under the Yocto Project umbrella but is maintained within its own governance structure.
The Application Development Toolkit (ADT) enables systems developers to provide software development kits (SDKs) for the distributions they create using the Yocto Project tools, providing applications developers a way to develop against the software stacks provided by those systems developers. The ADT includes a cross-compiling toolchain, debugging and profiling tools, and QEMU emulation and support scripts. The ADT also includes an Eclipse plug-in for those who like to work with integrated development environments (IDEs).
Several other tools under the Yocto Project banner are:
One vital component for any open source project is the community who develops and supports its components. The Yocto Project has an active community that is made up in part by organizations—including hardware manufacturers, operating systems and tools providers, and electronics manufacturers, thus capturing the entire spectrum of embedded development—and in part by individuals, many of whom work for those organizations.
The project is managed by a chief architect and a set of maintainers and technical leaders, who make all technical decisions. The member organizations, many of which normally compete with each other, collaborate to form an Advisory Board, whose role, as the name suggests, is advisory in nature. This board manages the resources for the project, including infrastructure, advocacy and outreach, and finance.
This section shows how to build a basic embedded Linux system using Poky, the reference build system. The process described here builds the reference distribution as well as all of the tools needed to build that distribution. If you prefer, you can also download pre-built binaries to avoid the need for compiling. Find details in the Yocto Project Quick Start Guide (see the resources at the right side).
The tools themselves are only supported on Linux platforms, specifically Ubuntu, Fedora, CentOS, and openSUSE. Note that the latest release of each of these systems is frequently not yet listed as being supported, though it often works anyway, and many other Linux distributions might also work.
If your host system does not run Linux, or if you’d just like to kick the tires without committing to installing anything, you can download the Yocto Project Build Appliance. This virtual machine has the latest official Yocto Project release pre-installed. The Build Appliance can run in either VMWare Player or VirtualBox, and it boots directly to the Hob GUI. If you use the Build Appliance, most of the following steps are already accomplished and you can skip to Test drive Hob. However, I recommend you still read the intervening sections so you understand what Hob is doing. Also, for the Hob documentation on the Yocto Project website, see the resources at the right side.
Download the Yocto Project tools with either of two methods. Make sure to use a disk that has at least 50GB free. 100GB is recommended.
You can download a tar file with the latest fully tested release from the Yocto Project downloads page. Download the tar file and then unpack it into a directory, as in Listing 1.
$ tar xjf poky-denzil-7.0.tar.bz2
$ cd poky-denzil-7.0
You can use git to grab the latest release (or any specific branch), although the master branch of development might be less stable than the tested release represented in the tar file. Listing 2 shows the use of git to download the latest bits.
$ git clone git://git.yoctoproject.org/poky.git
$ cd poky
Note that the subdirectory in this case is simply called poky without a release number, as it can be updated at any time using git.
To initialize your working environment, follow these steps:
First, install all the required development packages from your host system’s software repository. If you already develop software on your host, you might already have most of these installed. Listings 3, 4, 5, and 6 show how to install the needed packages on various distributions.
$ sudo apt-get install sed wget subversion git-core coreutils \
unzip texi2html texinfo libsdl1.2-dev docbook-utils fop gawk \
python-pysqlite2 diffstat make gcc build-essential xsltproc \
g++ desktop-file-utils chrpath libgl1-mesa-dev libglu1-mesa-dev \
autoconf automake groff libtool xterm libxml-parser-perl
$ sudo yum groupinstall "development tools"
$ sudo yum install python m4 make wget curl ftp tar bzip2 gzip \
unzip perl texinfo texi2html diffstat openjade \
docbook-style-dsssl sed docbook-style-xsl docbook-dtds fop xsltproc \
docbook-utils sed bc eglibc-devel ccache pcre pcre-devel quilt \
groff linuxdoc-tools patch cmake \
perl-ExtUtils-MakeMaker tcl-devel gettext chrpath ncurses apr \
SDL-devel mesa-libGL-devel mesa-libGLU-devel gnome-doc-utils \
autoconf automake libtool xterm
$ sudo yum -y groupinstall "development tools"
$ sudo yum -y install tetex gawk sqlite-devel vim-common redhat-lsb xz \
m4 make wget curl ftp tar bzip2 gzip python-devel \
unzip perl texinfo texi2html diffstat openjade zlib-devel \
docbook-style-dsssl sed docbook-style-xsl docbook-dtds \
docbook-utils bc glibc-devel pcre pcre-devel \
groff linuxdoc-tools patch cmake \
tcl-devel gettext ncurses apr \
SDL-devel mesa-libGL-devel mesa-libGLU-devel gnome-doc-utils \
autoconf automake libtool xterm
$ sudo zypper install python gcc gcc-c++ libtool fop \
subversion git chrpath automake make wget xsltproc \
diffstat texinfo freeglut-devel libSDL-devel
$ cd poky
$ . ./oe-init-build-env
BB_NUMBER_THREADS = "8"
PARALLEL_MAKE = "-j 8"
The initial build uses the host’s compiler to build the cross-compilation toolchain and any other needed build tools. BitBake also needs to download all the software packages, so it can take a while. When it is finished, you can find the resulting images in the tmp/deploy/images subdirectory.
$ ls tmp/deploy/images
Also, look in tmp/deploy/IMAGE/license.manifest. This is a list of all the licenses used in the entire image.
When the build is finished, you can boot it using QEMU emulating a 32-bit x86 machine. To do this, simply run:
$ runqemu qemux86
You can log in to the machine with the username root, with no password. Congratulations! You have built an embedded Linux distribution.
You can make many customizations to the final image, to the packages that make up the final image, and to the process itself. The build system is written in Python and is well documented.
Hob is a GUI that helps to automate some of the processes described earlier. The goal of Hob is to make the build process more visible and easier to understand. In reality, Hob is a front end for BitBake, and there is nothing you can do in Hob that you can’t also do (and script) from the command line. However, Hob shows the decision-making process graphically and outlines exactly which packages are built into the final image. Some of Hob’s other features include modifying existing images, saving customized images as templates, running images using QEMU, and deploying images to a Universal Serial Bus (USB) disk for live booting on a target device. Many other features are in the works for both functionality and usability.
To run Hob, simply type hob from your build directory, and the main screen appears. Start by choosing an architecture (in this case, qemux86). Hob then parses available recipes, as shown in Figure 1.
You can then choose a base image (the earlier example built core-image-minimal, so those binaries already exist). Choose core-image-minimal and then click View recipes or View packages. The screen in Figure 2 shows the scrolling lists in View packages. You can select or clear check boxes to include packages or remove them from the build, respectively. Click a column header to sort by that column.
On the Packages tab, note that Python is not included in the minimal build. You can add it by simply selecting the check box for python-2.7.2-r2.14, as in Figure 3. All the dependencies and sub-packages are immediately included as well.
Click Build image to see Hob build a new image with your choices. As the build progresses, you can watch the logs to see how things are going or click the Issues tab to see if any problems arise.
You can then run the resulting image in a QEMU emulator by simply clicking Run Image, as in Figure 4.
And now you can check to make sure Python was actually included in the build, as in Figure 5.
More information, tutorials, and videos are available on the Yocto Project website (see the resources at the right side).
Learning how to use the Yocto Project tools effectively can take some time, but it is time well spent. The flexible workflow, portable configurations, and high degree of configurability of the tools enables customization at every level of the build process. The Yocto Project represents a commercial workflow—it is designed and implemented by professional embedded systems developers. As of this writing, several operating system tools manufacturers are now refocusing their products based on the project. Take the tools for a spin, and if you like what you see, join the community.
Learn about the Linux perf command and how it can provide the same capabilities as OProfile tools, and much more
This article enables you to discover numerous tools for porting and tuning applications for Linux on Power- an integrated development…
IBM Power SystemsLinux+
January 24, 2019
Artificial IntelligenceData Science+
Back to top