Yocto Hardening: Read-Only Rootfs

You can find the other Yocto hardening posts from here!

Let’s continue the always-so-fun task of hardening the Yocto systems. This time, we will consider the file system integrity, specifically for the root file system. There is an earlier blog post about IMA & EVM that is somewhat related to this topic, but this time we will focus on the file systems themselves, and making them read-only, which is fortunately a bit simpler topic.

Read-Only File System

So, we have covered many hardening topics in this Yocto Hardening series. Still, there is one big shortcoming: we have not really considered how to prevent someone from simply removing the hardening measures. For example, if an adversary can simply rewrite a hardened configuration file, the entire effort of creating the hardened configuration is wasted. The first line of defence here is the traditional discretionary access control, where we set the read, write and execute permissions for all the files, preventing unwanted writes. However, this can be cumbersome and sometimes difficult to get right on larger systems and partitions.

Instead, it might be easier to bring out the Big Hammer and make everything on the file system read-only. This is not only useful from a security point of view, but it can also help prevent corruption of the file system. Ensuring the integrity of the root file system is extra important because it hosts most of the binaries and libraries in the system, and corruptions or undesired modifications there can result in your device becoming useless or vulnerable. It is quite common to make the root file system containing no actual secrets read-only and store other, more secret information on separate encrypted file systems. Note that using encryption does not guarantee data integrity, as those are separate concerns.

The read-only functionality on the system can be achieved on multiple levels:

  1. Physical (read-only switches, jumpers, one-time programmable memory, etc.)
  2. Kernel command line parameter for the root file system (ro / rw, with rw default)
  3. Mount options (ro mount option)
  4. File system itself (erofs, squashfs)
  5. Tuning the file system (tune2fs, at least ext can be tuned as ro)
  6. File attributes (chattr, ext has good support for these as well)
  7. File access controls (discretionary access control, mandatory access control)

This isn’t a hard categorisation; for example, the kernel command line parameter and mount options are more or less the same thing, they just do a similar thing at different parts of the boot. Another thing worth noticing is that some of the options are file system specific. While multiple file systems implement file system tuning and file attributes, not all of them do. Still, this list hopefully gives some idea of how files can be made read-only on Linux.

In this blog text, we’ll focus on points 2, 3, and 4. It is worth noting that everything except points 1 and 4 can be overridden by the root user if the system does not have mandatory access controls in place to prevent it. The file system can be remounted as rw, file system tunes can be removed, and file attributes and access controls can be modified. However, if there’s a physical mechanism preventing writes, or if the file system simply does not support writing, it should be possible to prevent writes on a running system.

But What If I Need to Write Something?

Often, we need to write some things to the system. For example, often there are /etc configurations to modify, and at least /var/lib is usually expected to be read-write, so a completely read-only system is rarely possible. There are a few different approaches that can be used to allow writes to the system.

First, we need to consider the persistence requirements of the written information. Is it important for the writes to persist, or do we just need to temporarily write something that can be discarded when the device shuts down? If there’s a need for persistence, usually another, non-volatile disk partition is required as the final target for the writes. If there is no such need, we can use tmpfs to write temporarily to RAM and forget everything once the device boots.

After this, the actual mechanism of making the sections of the file system writable needs to be decided. An overlayfs can be used, bind mounts can be used, or symlinks to a writable location can be used. Overlay file system is a transparent solution, and it is easy to wipe away writes for example for a factory reset. On the other hand, the first writes may be slow since the file needs to be copied up from the read-only partition to the read-write partition, and with larger files, this may be both time- and space-consuming. Overlay file systems are well-suited for situations where you need to make large areas writable, like entire /var/lib.

Bind mounts, on the other hand, simply map a writable directory on top of the directory in a read-only file system. This does not have the write overhead, but the writable directory has to be pre-populated with the existing content before mounting, and this can be time-consuming (and if the underlying file system changes for some reason, the writable directory has to be re-populated). Bind mounts are transparent for the processes that access them. They are best for situations where you need to make a single directory writable, or overlayfs is not an option for some reason.

Symlinks to a writable location are simple to implement, but they require that the processes accessing the files are smart enough to follow the symlinks. This is quite often the case, but can sometimes cause surprising failures. Symlinks can also be vulnerable if the programs utilising them are not careful, as the links or the files they point to may be modified by the attacker. Symlinks are best suited for making individual files writable, but I’d recommend using other options if possible.

Whichever option you choose, it is important to keep the ordering of the file system mounts correct. Also, it is important to ensure that the read-write functionality is ready before the services accessing the read-write locations launch, otherwise the services may try to write to read-only or non-existent locations.

Kernel Command-Line Parameter in Yocto

Let’s move on from theory to a more practical part. Linux provides a kernel command-line parameter that can be used to set the root file system read-only as soon as the kernel mounts it. The common reason for this is to allow fsck to check the root file system before the system properly kicks off and starts modifying it, but from the security point of view, it is also a good idea to keep the root file system ro at all times, even during the early initialisation. Quite commonly, the init process at some point may remount the rootfs read-write; more on that in the next chapter.

Unfortunately, there isn’t a simple way of adding a kernel command-line parameter in Yocto. Or there is, but how that is actually done depends a bit on your BSP. The most common variable to use is APPEND. However, at least the Raspberry Pi BSP uses CMDLINE instead. With QEMU, you can use QB_KERNEL_ROOT to define the whole root parameter, including the root file system options. Whatever you’ll have to use, you can simply add ro to the end of that variable in your build configuration to enable this.

Read-Only Rootfs in Yocto

Having a read-only root file system is quite a common requirement for embedded systems, and Yocto already provides a feature for this. To enable read-only rootfs support, all you (should) have to do is to add the following:

EXTRA_IMAGE_FEATURES:append = " read-only-rootfs "

This single feature does quite a lot of things; here are the ones I could see at least in Poky:

  • Add ro option to APPEND (i.e., add the read-only parameter to the kernel)
  • Ensure there are no on-target post-installation tasks as these are bound to fail
  • Mount the root file system as read-only in the user space, and add ro to fstab
  • Configure SSH host keys to be in a writable, usually volatile location
  • Create empty machine-id file in systemd systems

There may be more (or fewer) things, depending on the version of Yocto you’re using. Yocto also handles mounting /var/lib as a volatile read-write directory on read-only systems, as that is a directory that is expected to be writable. For sysvinit systems, there’s read-only-rootfs-hook.sh init script, and for systemd systems, there’s volatile-binds package. More on these later.

Needless to say, if your services are writing anything to the root file system, they’ll start failing after enabling this feature. Not all the packages in Yocto are compatible with it either, so you’ll most likely have to do some integration work.

File Systems in Yocto

The downside of these mount options is that the root user can quite easily remount the file systems as read-write. With a properly selected file system, we can prevent this. File systems like erofs and squashfs are designed to be read-only, making them great for ensuring file system integrity. From these two, erofs is optimised for read speeds, while squashfs is more geared towards optimising the compression. The correct choice depends on your use case.

To build either of the two (or some other) file systems, you can use IMAGE_FSTYPES variable. For example, to build an erofs image, you can do the following:

IMAGE_FSTYPES:append = " erofs"

How to actually take this built image into use depends on your system. If you’re using Wic to construct the firmware images, it is quite easy. For example, on Raspberry Pi, you can modify the Wic file in meta-raspberrypi to look like this to use erofs:

part /boot --source bootimg-partition --ondisk mmcblk0 --fstype=vfat --label boot --active --align 4096 --size 100
# Replace this:
# part / --source rootfs --ondisk mmcblk0 --fstype=ext4 --label root --align 4096
# with this:
part / --source rootfs --ondisk mmcblk0 --fstype=erofs --align 4096

Also, if you define the rootfstype in kernel command-line parameters (like meta-raspberrypi does), you need to update that as well. And last, but definitely not least, you’ll need to ensure that the kernel actually supports the root file system you’re using, because it does not get automatically enabled. Enabling erofs for example requires adding CONFIG_EROFS_FS=y in the kernel configuration.

Adding Read-Write Locations to Images

Finally, let’s consider how to add some writable locations to a Yocto image. First, we again need to consider where the writes should go and if the results need to be persistent or not. If we want the writes to be volatile, we don’t necessarily have to do anything to enable that. The default fstab in Yocto mounts a volatile tmpfs at /var/volatile, and we can use that as the target for the writes (and that is used as a target by the recipes presented soon).

On the other hand, if we want the writes to be persistent, we can add a new partition to the Wic-file as follows:

# Creates empty ext4 file system with the size of 4MB
# Note that ondisk parameter depends on your system,
# on Raspberry Pi it is at least mmcblk0
part /extra --size 4 --ondisk mmcblk0 --align 4096 --fstype=ext4 --label extra

Note that we also need to create the mount point for the partition, as that does not get generated automatically. One potential place for this is base-files, and it’s dirs755 variable. You could append the recipe with something like this to create the directory:

dirs755:append = " /extra"

Then, we can start mounting (or symlinking).

Systemd: volatile-binds

For the systemd systems, Yocto provides volatile-binds recipe that can be used to create volatile read-write mounts on a read-only root file system. You can quite simply define the mounts by appending the recipe with the following variable:

VOLATILE_BINDS = "\
    what where\n\
"

So if you’d want to mount /var/volatile/lib to /var/lib to make /var/lib writeable, it’d look like this (localstatedir expands to /var):

# I personally find the order confusing, but maybe
# that's just me
VOLATILE_BINDS = "\
    ${localstatedir}/volatile/lib ${localstatedir}/lib\n\
"

volatile-binds by default tries to first mount the locations as overlayfs, and if that fails, it does a bind mount. If you want to skip overlayfs and always go for the bind mount, set the AVOID_OVERLAYFS variable to 1.

Despite its name, volatile-binds recipe can, in theory, be used to mount non-volatile locations as well. However, the fallback functionality can be problematic for the persistent mounts. The information on the partition should be preserved, but mounting an overlay can fail semi-transparently and hide the existing data, leading to some “fascinating” debugging sessions. So, I’d recommend writing .mount files for the persistent mounts instead (I added a resource for creating .mount files to the end of this text).

Sysvinit: read-only-rootfs-hook.sh

For sysvinit systems, there’s less support. There’s read-only-rootfs-hook.sh init script that gets installed when a read-only rootfs is enabled. This script mounts /var/volatile/lib to /var/lib, first attempting overlayfs and then bind mount, but that’s all. For the rest of the mounts, you’ll have to figure out the mounting yourself. For persistent mounts, you could use fstab. For volatile mounts, you could copy the read-only-rootfs-hook.sh, and modify it with different parameters for the different mounts. Or generalise it a bit, create a script of it, and then create init scripts calling it. Still, with sysvinit, you’ll most likely have to do a bit more manual work.

Symlinks

For the sake of completeness, here’s how you can create a symlink from read-only rootfs to a writable location:

do_install:append(){
    ln -sf /extra/rw_file ${D}${sysconfdir}/rw_file
}

This option just consists of creating the links during the do_install phase of some recipe where it makes the most sense. Remember to ensure that the services do not start before the symlink target partition has been mounted. It is also worth noting that this does not initialise the actual file on the other partition, so the link is initialised broken.

Initialising the other partition requires either defining a new image that is copied to the extra partition in the Wic file, or creating a custom Wic plugin that populates the partition. These go outside the scope of this blog text, but if that’s something you’d like to know more about, leave a comment, and I’ll see what I can do.

Conclusion

This is where I copy-paste my typical “thanks for reading, hopefully you learned something new” ending. But, there’s a cliffhanger! These methods help prevent the changes to the file systems when the device is running, but what if someone powers it off and starts poking the storage then? That’s the moment when dm-verity becomes useful, and that’s the topic of the next blog post. At least I hope that it’ll be the next blog post. Until then!

As a reminder, if you’re interested you can find the previous Yocto hardening posts from here!

Recommended Reading

Tags:

Share
April 22, 2026 by Security

You may also like...

Leave a Reply

Your email address will not be published. Required fields are marked *