Even if you’ve built your Yocto distro from scratch, chances are you don’t fully know where every file in your image comes from. While it is practically impossible and quite unnecessary to know every single file in the system, having a rough understanding of the files and the packages that install them is important for two reasons:
- It helps to minimise the size of the file system. When we actually know what is in the file system, it is easier to see the unnecessary packages and remove them. Something like this comes in handy when updating the version of Yocto, as the package sizes tend to increase.
- It makes the system more secure. When considering the attack surface of a Linux system, almost every file is a potential attack vector. Therefore, we want to understand what is actually in the image and why it is there. If there is no good reason to have the file on the system, it should be removed.
In embedded Linux systems, you usually should not have binaries and libraries “in case you happen to need them”. There should be enough understanding of the system to know what is actually needed and what is not, and the system should be tailored to this need.
So, to begin understanding your system, you can log into it and start browsing the usually exciting locations like /usr/bin or /usr/lib for files. Some of the files and their origin and usefulness may be obvious, while others may lead to confusion. Let’s go through the two core questions that come to mind when browsing the files in the system: who installs this, and more importantly, why?
Who Installs This File?
First, we try to figure out which Yocto package is responsible for installing a given file in the file system. Granted, this isn’t always that difficult to guess. For example, the fact that libxml2.so gets installed by libxml2 is hardly shocking. However, figuring out who installs, for example, /usr/bin/unicode_stop binary isn’t as obvious (the answer is kbd).
Fortunately, openembedded-core has a solid collection of scripts, and for this specific question there’s one that’s especially useful: oe-pkgdata-util. The script has a few different commands that can be used to inspect the package data of the recipes. This package data contains, among many other things, the installed paths and files. To figure out who installs some file, we can use the find-path command. Let’s use /usr/bin/ssh-keygen as an example:
$ ./openembedded-core/scripts/oe-pkgdata-util -p build/tmp/pkgdata/qemux86-64 find-path /usr/bin/ssh-keygen
openssh-keygen: /usr/bin/ssh-keygenThis gives us the package name, openssh-keygen, which helps us go forward. Since we are trying to figure out who installs the file, we are primarily interested in the package name. However, quite often we need the recipe as well, for example, to figure out how the package contents are defined. For that, we can use the following lookup-recipe command:
$ ./openembedded-core/scripts/oe-pkgdata-util -p build/tmp/pkgdata/qemux86-64 lookup-recipe openssh-keygen
opensshThis gives the recipe name. To find the meta-layer containing the recipe file, we can use bitbake-layers command:
$ bitbake-layers show-recipes openssh
=== Matching recipes: ===
openssh:
meta 10.3p1A few things that should be kept in mind here: the package and recipe names may differ quite a lot from each other, and you shouldn’t make strong assumptions about the package contents based on the package name. So, you should always check the recipe to see how the packages are defined. Also, remember to check the bbappends as well with bitbake-layers show-appends because they may modify the packages.
Another thing worth mentioning: oe-pkgdata-util does not handle symlinks that are installed with update-alternatives, as those are handled outside the package data. In this case, the package installs a file named <COMMAND_NAME>.<PACKAGE_NAME>, to which the alternative name gets linked. In these cases, investigating the symlinks themselves should help figure out who installs the target binary. For example, pidof in /bin on a sysvinit system:
# We can see that sysvinit pidof is active
lrwxrwxrwx. 1 root root 19 Apr 5 2011 pidof -> /bin/pidof.sysvinit
-rwxr-xr-x. 1 root root 23K Apr 5 2011 pidof.procps
lrwxrwxrwx. 1 root root 14 Apr 5 2011 pidof.sysvinit -> /sbin/killall5Searching for path /bin/pidof will yield no results, because it is created during the installation phase. Instead, you’ll have to search for /bin/pidof.procps or /bin/pidof.sysvinit.
If you get no results with the oe-pkgdata-util, it’s possible that the file is generated in a package postinstall hook or during runtime. In these scenarios, you could first try grepping the meta-layers, and if that fails, ultimately try grepping the sources.
One more potentially useful oe-pkgdata-util command before moving on. You can use list-pkg-files command to see the files installed by a certain package if you want to inspect the contents. Here’s the output for openssh-sshd as an example:
$ ./openembedded-core/scripts/oe-pkgdata-util -p build/tmp/pkgdata/qemux86-64 list-pkg-files openssh-sshd
openssh-sshd:
/etc/default/volatiles/99_sshd
/etc/init.d/sshd
/etc/pam.d/sshd
/etc/ssh/moduli
/etc/ssh/sshd_config
/etc/ssh/sshd_config.d/50-selinux.conf
/etc/ssh/sshd_config_readonly
/usr/libexec/openssh/sshd_check_keys
/usr/libexec/sshd-auth
/usr/libexec/sshd-session
/usr/sbin/sshdThen, to the second question.
Why Is The File Installed?
After we have found the package that installs the file to the system, we can start to wonder why that package is installed in the first place. This isn’t always quite as straightforward as one would hope, because there are plenty of ways for the package to end up in the image.
IMAGE_INSTALL
The first obvious step is checking the IMAGE_INSTALL variable to see if the package is listed there. bitbake-getvar IMAGE_INSTALL -r <IMAGE_NAME> can be used for this. If the package is in the variable, congratulations, you got the easy solution to the problem. If not, the package is usually pulled in as a dependency or a recommendation (or as a part of a packagegroup). Let’s check these out next.
RDEPENDS & RRECOMMENDS
For figuring out the dependencies, BitBake has a useful feature named build history. Enabling build history allows you to easily go through the runtime dependencies for the packages, among many other things. Runtime dependencies (RDEPENDS) are important because they are the ones that actually get installed on the system. You can enable the build history with the following build configuration:
INHERIT += "buildhistory"
BUILDHISTORY_COMMIT = "1"After the next build, a new buildhistory directory should appear under the build directory. The contents are documented in depth in the documentation, but in the context of this blog text, we are interested in the packages subdirectory. Underneath that, we can find the recipes and packages for each target triplet. After generating the build history, we need to check the RDEPENDS value for each package and see if it contains the target package we’re looking for. Here’s an example script that checks who depends on openssh-keygen:
# Yes, AI generated this script.
# I couldn't come up with something like this even if I tried.
# Run the command in the root of the TOPDIR, above buildhistory directory
$ PKG=openssh-keygen; grep -lE "^RDEPENDS[[:space:]]*=.*(^|[[:space:]])${PKG}([[:space:]]|\(|$)" buildhistory/packages/*/*/*/latest
buildhistory/packages/x86-64-v3-sulka-linux/openssh/openssh/latest
buildhistory/packages/x86-64-v3-sulka-linux/openssh/openssh-sshd/latestFrom this output, we can see that openssh and openssh-sshd have a runtime dependency on openssh-keygen. You may also want to check the runtime recommendations, as the recommendations are installed by default if they’re not disabled. To do that, replace RDEPENDS with RRECOMMENDS in the command above.
Then, the packagegroups. Packagegroups are recipes that depend on and recommend multiple packages, usually to provide some bigger functionality. They define the packages they want using RDEPENDS and RRECOMMENDS as well. However, instead of going through the build history, it makes more sense to directly query the important variables with bitbake-getvar. Here’s an example with packagegroup-core-boot:
$ bitbake-getvar RDEPENDS:packagegroup-core-boot -r packagegroup-core-boot
RDEPENDS:packagegroup-core-boot="base-files base-passwd busybox busybox-hwclock modutils-initscripts initscripts netbase busybox sysvinit udev update-alternatives-opkg v86d"
$ bitbake-getvar RRECOMMENDS:packagegroup-core-boot -r packagegroup-core-boot
RRECOMMENDS:packagegroup-core-boot="syslog-ng init-ifupdown"DEPENDS
If these yield no results, you could check the build dependencies next. Typically, the shared libraries that the programs link against get installed to the system even if they are not listed in the RDEPENDS. To check the build dependencies, you can use oe-depends-dot script. This requires the task dot file for the image you want to investigate. This file can be generated with the -g option:
$ bitbake <IMAGE_NAME> -gAfter this, you can run the script with -k <RECIPE_NAME> and -w parameters. The script will output all the recipes that depend on the recipe you pass as the parameter, and the whole dependency chain. Here is a sample of libxml2 recipe output on my build:
$ ./openembedded-core/scripts/oe-depends-dot -k libxml2 -w build/task-depends.dot
Because: core-image-base shared-mime-info glib-2.0 systemd alsa-utils dbus packagegroup-core-boot pciutils syslog-ng uutils-coreutils libpam selinux-init selinux-labeldev packagegroup-selinux-minimal packagegroup-base cronie kbd libcap passwdqc policycoreutils shadow sudo util-linux e2fsprogs run-postinsts packagegroup-sulka sysstat alsa-state
core-image-base -> syslog-ng -> glib-2.0 -> shared-mime-info -> libxml2
core-image-base -> packagegroup-base -> packagegroup-core-boot -> syslog-ng -> glib-2.0 -> shared-mime-info -> libxml2
core-image-base -> packagegroup-base -> alsa-state -> alsa-utils -> systemd -> shared-mime-info -> libxml2
core-image-base -> dbus -> systemd -> shared-mime-info -> libxml2
core-image-base -> packagegroup-base -> pciutils -> systemd -> shared-mime-info -> libxml2
core-image-base -> sysstat -> cronie -> shadow -> util-linux -> libpam -> uutils-coreutils -> systemd -> shared-mime-info -> libxml2
core-image-base -> packagegroup-base -> packagegroup-sulka -> sysstat -> cronie -> shadow -> util-linux -> libpam -> uutils-coreutils -> systemd -> shared-mime-info -> libxml2
core-image-base -> e2fsprogs -> util-linux -> libpam -> uutils-coreutils -> systemd -> shared-mime-info -> libxml2
core-image-base -> run-postinsts -> util-linux -> libpam -> uutils-coreutils -> systemd -> shared-mime-info -> libxml2
core-image-base -> kbd -> libpam -> uutils-coreutils -> systemd -> shared-mime-info -> libxml2
core-image-base -> libcap -> libpam -> uutils-coreutils -> systemd -> shared-mime-info -> libxml2
core-image-base -> packagegroup-sulka -> passwdqc -> libpam -> uutils-coreutils -> systemd -> shared-mime-info -> libxml2
core-image-base -> policycoreutils -> libpam -> uutils-coreutils -> systemd -> shared-mime-info -> libxml2
core-image-base -> packagegroup-base -> packagegroup-selinux-minimal -> selinux-init -> policycoreutils -> libpam -> uutils-coreutils -> systemd -> shared-mime-info -> libxml2
core-image-base -> packagegroup-selinux-minimal -> selinux-labeldev -> policycoreutils -> libpam -> uutils-coreutils -> systemd -> shared-mime-info -> libxml2
core-image-base -> packagegroup-sulka -> sudo -> libpam -> uutils-coreutils -> systemd -> shared-mime-info -> libxml2As mentioned, these are build-time dependencies, not run-time dependencies, so they are mostly applicable to the shared libraries. Note that not all the build dependencies are necessarily library dependencies. Also note that here we are using recipe names. -k in the command stands for key, and -w stands for why. If you want to reverse the output and see the dependencies of some recipe, you can use -d instead of -w:
$ ./openembedded-core/scripts/oe-depends-dot -k libxml2 -d build/task-depends.dot
Depends: zlib glibc gcc-runtime libtool-cross pseudo-native dwarfsrcfiles-native automake-native binutils-cross-x86_64 gcc-cross-x86_64 libtool-native quilt-native xz-native patch-native pkgconf-native rpm-native autoconf-nativeBoth are useful in different situations. If you’re trying to understand who requires some library, use the first command. If you’re trying to figure out what libraries some program needs, use the second command. Note that not all the dependencies listed by the second command get installed to the final system; only the packages containing the shared libraries are installed, and only if they’re needed.
Machine and Distro Dependencies
If nothing has helped so far, you could next try checking out the machine and distro requirements (and recommendations) that may get installed. Use bitbake -e <IMAGE_NAME> to get the environment, and then check if these variables contain something you’re looking for:
- MACHINE_ESSENTIAL_EXTRA_RDEPENDS
- MACHINE_ESSENTIAL_EXTRA_RRECOMMENDS
- MACHINE_EXTRA_RDEPENDS
- MACHINE_EXTRA_RRECOMMENDS
- DISTRO_EXTRA_RDEPENDS
- DISTRO_EXTRA_RRECOMMENDS
IMAGE_FEATURES
And, if that wasn’t enough, IMAGE_FEATURES can also install packages. Check the core-image.bbclass for the list of the image features, and how they define the packages they install (ssh-server-dropbear is a simple example). You should also be able to check FEATURE_INSTALL variable to see what packages are being installed by the image features, but note that it is an internal variable and you should not modify it yourself.
Closing Words
Thanks for reading; hopefully you found this informative. I also hope that I didn’t forget some obvious way of installing packages to the image, as there are quite a few ways to get some bytes in that final blob. This text should cover at least the most common methods, so if there’s something else, then it’s something quite esoteric. See you in the next text!
Recommended Reading Watching
This time I actually don’t have anything to read, but two presentations to watch. They are about root file system size reduction, but it is closely related to what’s presented in this text, so I think these are relevant here. Sergio Prado’s presentation is especially relevant for the topic of this post, as it goes through build history in more detail than what was written here.
Yocto Project Virtual Summit 2020: Size reduction techniques with Yocto Project – Khem Raj
