Yocto Hardening: Firewalls, Part 1: nftables

Find all of the Yocto hardening texts from here!

The eternal task of making the Yocto Linux build an impenetrable fortress continues. Next, we’ll look into setting up a firewall two different ways: the easy way with nftables, and in the part two the easily configurable way with firewalld. I must warn you beforehand that this text contains a bit of resentment towards kernel configuration. I also used ChatGPT as a tool for writing a part of this text, but I’ll inform you of the section I asked it to write.

So yeah, firewall. The system keeping the barbarians out of our Linux empire. Also, one of the most bad-ass names for a program, at least if taken literally. Reality is a bit less exciting, turns out “firewall” is some technical construction term. In the magical world of computers, a firewall is a piece of software used to allow or deny network traffic coming in and out of a machine. It’s mighty useful for improving the overall security of the system to be able to prevent unwanted remote connections and such.

Step 0 – Configuring Kernel

The first step of installing and configuring a firewall on a Yocto image is ensuring that the kernel is configured correctly. What, do you think you can just start configuring the firewall and blocking your nefarious enemies just like that? Pfft, get out with that casual attitude (please don’t).

To get some basic traffic filtering with nftables done, we need to ensure that the Netfilter is enabled in the kernel, as well as the Netfilter support for nftables, and that some nftables modules are installed as well. It’s all a bit confusing, but to summarize: Netfilter is the kernel framework for packet filtering and nftables is the front-end for Netfilter that can be used from user space. nftables can then be built with or without modules to extend its functionality. The following kernel configuration fragment should do the trick:

Getting this config fragment working was where things got problematic for me. I thought that I’d just slap =y to everything and be done with it, no need to install and load modules. Well, the configuration process of the kernel consists of merging the defconfig, kernel features, and configuration fragments, and as it turns out, the kernel feature enabling the Netfilter overrode some parts of my config fragment, changing =y to =m. For some reason bitbake didn’t give a warning about this. I’m quite sure it used to do.

But where is this mythical Netfilter kernel feature located? The fragment above enables only nftables related things, how to get the Netfilter working? Well, Kirkstone release (3.1.27) of Poky has this line that enables the feature by default in the kernel, and the related configuration fragment looks like this. The multiple configs lead to funny situations where you try to remember the difference between CONFIG_NF_CONNTRACK and CONFIG_NFT_CT and where they are defined and to what value.

Ensure that you’re also installing the kernel modules to the image by adding the following line somewhere in the image configuration:

IMAGE_INSTALL:append = " kernel-modules"

No use enabling the modules if they’re not installed, and by default nothing installs them, so they won’t get added to the final image. At least to the qemu64 core-image-minimal image they won’t be installed by default, guess if I found that out the hard way as well. If you need more fine-grained control over the modules that get installed, you can add them as a runtime recommendation to some package:

RRECOMMENDS:${PN} += "kernel-module-nf-tables kernel-module-nf-conntrack kernel-module-nft-ct ...

Most likely nftables package is the best place for this. However, I’d advise against this approach if possible, because there will be a lot of modules later on. If you choose to go down this route, the naming format of the modules is fortunately quite self-explanatory. CONFIG_NF_TABLES being built as a module generates kernel-module-nf-tables package, CONFIG_NFT_CT results in kernel-module-nft-ct being built etc, so this should be easily scriptable (although painful to maintain).

The Easy Firewall – nftables

The easy way of getting some firewalling done involves installing nftables, adding some rules to it, and loading them as a part of the start-up. This is suitable if you have a few interfaces with a simple configuration that doesn’t change often, preferably ever.

The actual first step of getting the firewall up and running is adding the nftables package to the image if you don’t have it already installed. This can be easily checked by running nft command. If it fails, add the package to the image configuration:

IMAGE_INSTALL:append = " nftables"

If you see the following types of errors when running the nft command it means that your kernel configuration is missing either nftables support (the first error) or some nftables module (the second error):

root@qemux86-64:~# nft
../../nftables-1.0.2/src/mnl.c:60: Unable to initialize Netlink socket: Protocol not supported
root@qemux86-64:~# 

root@qemux86-64:~# nft -f /tmp/test.conf
/tmp/test2:32:9-16: Error: Could not process rule: No such file or directory
        ct state vmap { established : accept, related : accept, invalid : drop } 
        ^^^^^^^^
root@qemux86-64:~# 

Quite often the firewall in Yocto is empty by default. The active rules of the firewall can be checked by running nft list ruleset command to print the current ruleset. If nothing is output, no rules are present. If something gets output, there are some rules present. Here’s a short reference of commands on how to set some basic rules and list them on the command line.

# List ruleset
nft list ruleset

# Add a table named filter for IP traffic 
nft add table inet filter

# Add an input chain that drops traffic by default
nft add chain inet filter input \{ type filter hook input priority 0 \; policy drop \}

# Add rule to allow traffic to port 22
nft add rule inet filter input tcp dport \{ 22 \} accept

# List tables and rules in a table
nft list tables
nft list table inet firewall

In addition to listing the tables and rulesets from inside the device, you should also try a black-box approach. Running nmap command against your device is a good way to check how the outside world sees your device. In this example, I have added the Dropbear SSH server to the image. Because the server is listening on port 22 and there are no rules defined in the firewall, the port is seen as open:

esa@ubuntu:~$ nmap 192.168.7.2
Starting Nmap 7.80 ( https://nmap.org ) at 2023-07-22 16:14 UTC
Nmap scan report for 192.168.7.2
Host is up (0.0024s latency).
Not shown: 999 closed ports
PORT   STATE SERVICE
22/tcp open  ssh
Nmap done: 1 IP address (1 host up) scanned in 0.06 seconds

The command line commands are useful for editing the configuration run-time, but for the rest of the text we’re going to focus on a file-based configuration that is set during the Yocto build. The configurations presented here are mostly based on “a simple ruleset for a server” in nftables wiki. It contains some useful extra things not presented here, like a rule for the loopback interface, differentiating between ipv4 and ipv6 traffic and logging, and I recommend checking it out.

Usually, a good way to start creating a firewall ruleset is to block everything by default, and then start poking holes as needed. To block everything, you can write a configuration file with the following content:

flush ruleset

table inet firewall {
    chain inbound {
        type filter hook input priority 0; policy drop;
    }

    chain forward {
        type filter hook forward priority 0; policy drop;
    }

    chain output {
        type filter hook output priority 0; policy drop;
    }
}

This config creates one rule table named firewall for inet traffic. The table contains three chains: inbound, forward and output. You can load the ruleset with nft -f <config_file> command. Once you’ve blocked all traffic, you can give the nmap command another go. As you can see, blocking is really effective. nmap actually considers the device to be non-existent at this point:

esa@ubuntu:~$ nmap 192.168.7.2
Starting Nmap 7.80 ( https://nmap.org ) at 2023-07-22 16:18 UTC
Note: Host seems down. If it is really up, but blocking our ping probes, try -Pn
Nmap done: 1 IP address (0 hosts up) scanned in 3.03 seconds

Trying to ping from the device itself to the outside world also fails because the output is blocked. In theory, allowing outgoing traffic is not the worst idea, usually many sample configurations actually allow it. But if you know the traffic that’s going to go out and the ports that will be used there’s in my opinion no sense allowing all of the unnecessary traffic. That’s just an attitude that will lead to botnets. But if you want/need to allow all outgoing traffic, you can just drop the output chain from the config. By default everything will be accepted.

After nothing goes in or out, it’s time to start allowing some traffic. For example, if you want to allow SSH traffic from the barbarians to port 22 (generally a Bad Idea, but for the sake of an example), you can use the following ruleset:

flush ruleset

table inet firewall {
    chain inbound {
        type filter hook input priority 0; policy drop;

        # Allow SSH on port TCP/22 for IPv4 and IPv6.
        tcp dport { 22 } accept
    }

    chain forward {
        type filter hook forward priority 0; policy drop;
    }

    chain output {
        type filter hook output priority 0; policy drop;

        # Allow established and related traffic
        ct state vmap { established : accept, related : accept, invalid : drop } 
    }
}

If you want to define multiple allowed destination ports, separate them using commas, like { 22, 80, 443 }. If you choose not to block outgoing connections, you obviously don’t need the output chain presented here.

On the other hand, if there’s a process in the Yocto image that wants to contact the outside world, you could consider adding a ruleset like below:

flush ruleset

table inet firewall {
    chain inbound {
        type filter hook input priority 0; policy drop;

        # Allow established and related traffic
        ct state vmap { established : accept, related : accept, invalid : drop } 
    }

    chain forward {
        type filter hook forward priority 0; policy drop;
    }

    chain output {
        type filter hook output priority 0; policy drop;

        # You can check the ephemeral port range of kernel 
        # from /proc/sys/net/ipv4/ip_local_port_range
        tcp sport 32768-60999 accept
    }
}

Note how we need to use a port range for the output. Outgoing connections usually use a port from an ephemeral range for their connection, meaning that you may not know beforehand the exact port that needs to be allowed. Having fixed ports for outgoing traffic makes the device more susceptible to port scanning and prevents multiple connections from the same service because the port is already used, but if you choose to create a service that uses a static source port, you can use a single port number in the rule.

You can combine the rules in the chains as required by your system and services. If you want to enable the ping command (for ipv4), you can add the following piece of configuration to the input chain (note that this is just a single line of configuration, not a full config):

icmp type echo-request limit rate 5/second accept

In the end, creating the firewall ruleset is fairly simple: block everything and allow only necessary things. The difficulty really comes from keeping the rules up-to-date when something changes. However, when developing an embedded device with Yocto you generally should have a well-defined list of allowed ports & traffic (as opposed to a general-purpose computer used by a living human where all sorts of traffic and configs may come and go at the user’s whim). If that’s not the case, the part 2 where we work with firewalld may be useful to you.

So, now we know how to write the perfect configuration file. But how to add this to a Yocto build? You can create a bbappend file for nftables into your own meta-layer. This append then installs the configuration file, because nftables does not install any configuration by default. An example of how to add a configuration to nftables can be found in meta-firewall-examples repository I made for this blog post.

You should also note that nftables package doesn’t contain any mechanism to activate the rules during the start-up. Therefore you should append the nftables recipe so that it adds an init.d script or a service file that activates the rules from the configuration file before the networking is started. You can find an example of this as well from the same meta-firewall-examples repository.

Note that if you do run-time edits to the configuration with nft command, the changes will be lost when the device is reset. To ensure that the changes are kept, you can run the following command to overwrite the default config with the new, enhanced configuration:

nft list ruleset > /path/to/your/configuration-file

And one more thing: if you face some No such file or directory issues when trying more exotic configurations you are most likely missing some kernel configurations. In part 2, we’ll be enabling pretty much every feature of the Netfilter, so that’ll (most likely) be helpful. On that cliffhanger, we eagerly await the next chapter in this ever-evolving journey of discovery and innovation (I asked Chat-GPT to write that sentence).

Tags: firewallhardeninglinuxsecurityyocto