Module Signing Keys (Without Building Kernel)

I posted a blog post about kernel module signing in Yocto last week. I received a good question related to that blog post: how does module signing work in a situation where the entity distributing a kernel binary is different from the entity creating kernel modules? How to handle secrets in this kind of scenario? Or at least I understood the question that way. This will be generic Linux stuff, so no Yocto knowledge required this time.

In this situation, we’ll assume that the kernel distributor only distributes a pre-built binary of the kernel, not the sources. If you have the sources, you should be able to compile the kernel with the certificates yourself, and there really isn’t any problem in the first place. (IANAL, but I think not sharing sources goes against the license of the Linux kernel, but let’s play along with this totally fictional situation where a vendor does not care about the licenses and legal responsibilities.) I’m using terms kernel maintainer and module developer, but you can also think of these as kernel distributor and kernel user if that makes more sense.

So, how to get the certificates into a pre-built kernel? There are (at least) two ways to do this.

Option 1: Adding keys during runtime

The first option is loading the module signing keys to the kernel during runtime. This can be done with the keyctl command. When loading keys during runtime, we obviously cannot add the keys to the built-in keyring. We have to use the secondary keyring of the kernel instead. Keys can be added to this keyring if the key being added is vouched for by a key in the built-in keyring, or by a key already in the secondary keyring. In our scenario, this means that the kernel maintainer has to build a trust anchor certificate authority to the kernel (CONFIG_SYSTEM_TRUSTED_KEYS), and enable the secondary keyring in the kernel configuration (CONFIG_SECONDARY_TRUSTED_KEYRING).

To get their keys loaded, the module developer first has to get their own certificate authority signed by the kernel vendor trust anchor CA. The module developer should use an intermediate certificate authority instead of directly getting their module signing keys signed, because that way they can generate and rotate the keys as needed without requiring a signature from the kernel maintainer every time. Also, signing the module signing keys with the intermediate certificate authority ensures that the keys are trusted by the module developer.

One of the key benefits of asymmetric cryptography becomes apparent in this scenario: the chain of trust between the kernel maintainer and the module developer can be established without either party having to share their secrets. The module developer sends a certificate signing request containing only public information, the vendor signs it with their trust anchor CA private key, and returns the signed certificate. Both parties stay in control of their secrets.

After the kernel vendor has signed the CSR, the module developer can generate the module signing keys and sign them with their intermediate CA. Once that is done, the developer can then add the intermediate certificate authority and the module signing keys to the secondary keyring in Linux. The intermediate CA has to be loaded first, because the secondary keyring requires that someone vouch for the key. The trust anchor in the built-in keyring vouches for the intermediate CA after it has been signed. The result should look something like this:

First, let’s print the built-in keyring contents, and then add the intermediate CA to the secondary keyring. You can see from the output that I’m still using the same example keys from the Yocto blog post:

test@qemux86-64:~$ sudo keyctl show -x %:.builtin_trusted_keys
Keyring
0x2cb8cd4a ---lswrv      0     0  keyring: .builtin_trusted_keys
0x24425f93 ---lswrv      0     0   \_ asymmetric: example.com: meta-intel-iot-security example signing key: 460501e7a2143d52e3abaeb1d24d378d824bff54
0x153f620e ---lswrv      0     0   \_ asymmetric: example.com: meta-intel-iot-security example certificate signing key: e16733db10cf8d84f5b52aeafe18daead1746c0d

test@qemux86-64:~$ cat /tmp/intermediate-ca.der | sudo keyctl padd asymmetric "module ca" %:.secondary_trusted_keys

After the intermediate CA has been loaded, the module signing keys can be loaded into the secondary keyring, and we can print out the secondary keyring contents:

test@qemux86-64:~$ cat /tmp/module-key.der | sudo keyctl padd asymmetric "module key" %:.secondary_trusted_keys

test@qemux86-64:~$ sudo keyctl show -x %:.secondary_trusted_keys
Keyring
0x10db4831 ---lswrv      0     0  keyring: .secondary_trusted_keys
0x2cb8cd4a ---lswrv      0     0   \_ keyring: .builtin_trusted_keys
0x24425f93 ---lswrv      0     0   |   \_ asymmetric: example.com: meta-intel-iot-security example signing key: 460501e7a2143d52e3abaeb1d24d378d824bff54
0x153f620e ---lswrv      0     0   |   \_ asymmetric: example.com: meta-intel-iot-security example certificate signing key: e16733db10cf8d84f5b52aeafe18daead1746c0d
0x01572d40 --als--v      0     0   \_ asymmetric: module key
0x2bd2c6fa --als--v      0     0   \_ asymmetric: module ca

From the output, it can be seen how the secondary keyring contains the built-in keyring. After loading the module signing keys, it should be possible to probe modules signed with either the built-in keys or the new keys added during runtime.

Option 2: Adding keys between build and runtime

This kind of runtime management of the keys creates complexity in the system. Also, it adds extra responsibility to the kernel maintainer, as they need to manage the trust anchor in the kernel and sign the intermediate certificate authorities for the developers. An alternative for managing the keys during runtime is adding them to the kernel binary after it has been built. This, in practice, pushes all the responsibility to the module developer. Note that the kernel supports adding one extra certificate post-build, which means that the module signing key still has to be handled during runtime (or the module developer adds just the module signing key to the built-in keyring). The keyring content should end up looking like this:

Linux configuration has CONFIG_SYSTEM_EXTRA_CERTIFICATE option that allows reserving space for one extra certificate in the built kernel binary. The extra certificate can be loaded with an aptly named insert-sys-cert script. The script takes the kernel symbol table (System.map), kernel binary (vmlinux) and the desired certificate in der format as parameters:

esa@pc-esa:~/$ linux-qemux86_64-standard-build/scripts/insert-sys-cert -s linux-qemux86_64-standard-build/System.map -b linux-qemux86_64-standard-build/vmlinux -c ~/intermediate-keys/intermediate-ca.der

After embedding the additional certificate into the built-in keyring, keyctl outputs the following for built-in keyring:

test@qemux86-64:~$ sudo keyctl show -x %:.builtin_trusted_keys
Keyring
0x2a1c87a0 ---lswrv      0     0  keyring: .builtin_trusted_keys
0x02047894 ---lswrv      0     0   \_ asymmetric: example.com: meta-intel-iot-security example signing key: 460501e7a2143d52e3abaeb1d24d378d824bff54
0x3e7a36f5 ---lswrv      0     0   \_ asymmetric: Developer Intermediate CA: 5cb52356157fda631a26d10974e8221d0779d234
0x1186d462 ---lswrv      0     0   \_ asymmetric: example.com: meta-intel-iot-security example certificate signing key: e16733db10cf8d84f5b52aeafe18daead1746c0d

Loading the module signing key happens the same way as it did in option 1, and after adding the key, the secondary keyring looks like the following:

test@qemux86-64:~$ sudo keyctl show -x %:.secondary_trusted_keys
Keyring
0x3557e98c ---lswrv      0     0  keyring: .secondary_trusted_keys
0x3c6a8ccf ---lswrv      0     0   \_ keyring: .builtin_trusted_keys
0x011f1f9a ---lswrv      0     0   |   \_ asymmetric: example.com: meta-intel-iot-security example signing key: 460501e7a2143d52e3abaeb1d24d378d824bff54
0x339f14a4 ---lswrv      0     0   |   \_ asymmetric: Developer Intermediate CA: 5cb52356157fda631a26d10974e8221d0779d234
0x02414810 ---lswrv      0     0   |   \_ asymmetric: example.com: meta-intel-iot-security example certificate signing key: e16733db10cf8d84f5b52aeafe18daead1746c0d
0x19283ded --als--v      0     0   \_ asymmetric: module key

About as expected. As mentioned, we still cannot completely get rid of the runtime key management, because the module signing key has to be loaded manually, but at least the certificate authority is now part of the kernel binary and cannot be easily replaced. The downside is that if the intermediate CA private key leaks, we cannot simply rotate it; we need to update the whole kernel.

Thanks for reading, that’s all for this quick post. Here’s one more meme for making it this far:

Tags: keyslinuxsecurity