virtual machines

Read the previous parts of “Yocto and WSL” series:
Yocto? On WSL2? Easier than you think!
Yocto and WSL, part 2: The Graphic Boogaloo

The ultimate showdown of the ultimate destiny!

All the good things are trilogies. Star Wars original trilogy, Nolan’s Batman trilogy, and The Hobbit trilogy. And now, Yocto & WSL trilogy. As per usual with the trilogies, the last part may be a bit of a letdown for the hardcore fans. This time we’re not doing really anything technical or exciting. Instead, we’re comparing some numbers while trying to decide if this whole exercise was worth it or not.

To judge the usefulness of WSL in the Yocto build context were doing a comparison between WSL machine and a “regular” virtual machine to see which one is actually better for building Yocto (if you’re forced to use Windows). Better means faster in this case. For this purpose, I created a VMWare virtual machine that matched my WSL machine’s specs: 4 cores, 8GB, and enough disk space to accommodate the build area. For the operating system I chose Ubuntu Server 20.04.4, and as a reference build target I used the simplest core-image-minimal recipe with both machines.

As a bit of an extra, I’ll be adding a comparison to a cloud build server as well for the same price! For this purpose, I prepared a similar 4-core 8GB build server on Hetzner. This is a virtual machine as well, but it’s interesting to have more things to compare to. Especially because the cloud instances provide an option to scale up in case they aren’t powerful enough. Also, like WSL and VMWare, they can be used as an alternative for a Linux build machine. Without further ado, let’s get into crunching numbers.

First, to actually make a worthwhile comparison between different build times, I’m going to separate the process of downloading the sources from the actual build itself. I did three runs of downloads because “third time’s the charm”. VMWare was using a NAT network connection here. The separation was done because the time it takes to download the sources depends on the load on the source code servers, how much my neighbor happens to be using the internet at the given moment, and the alignment of the stars. Or what do you think of these results:

There were two packages that became bottlenecks in these tests: cross-localedef-native and linux-yocto. Especially the former. Other packages were downloaded in a few minutes but cross-localedef-native took almost always at least half an hour to download. In the WSL run #3 cross-localedef-native took only about 15 minutes to download, and instantly the whole download process was a lot quicker. In general, I would still boldly claim that Hetzner build is potentially the fastest on average. Or at least it would make sense, as the Internet pipes in their data centers are most likely wider than mine at my home.

Then, the actual build times. For these, I actually did four runs, because “third time’s the charm and one more for a good measure” (this blog text isn’t quite as scientific as I make it out to be). As mentioned earlier, I built the core-image-minimal for all these runs. These builds were done without downloading the sources in an attempt to get the build times to be a bit more stable. Here’s the table:

As perhaps expected, the virtual machine was constantly the slowest option. What however is slightly surprising is that the WSL was almost as fast as the cloud build server with the same resources. Or actually, with better resources as the build server had an SSD disk while WSL was running on an old HDD I bought from a friend of mine for 30 euros some years ago.

But yeah, in the light of this evidence I’d say that if you’ve got some dollars to spare, getting a cloud build server is the best option for building Yocto (if you can’t get a native Linux build machine that is). It’s not only the fastest option, but it’s also scalable, so when you’re building something else than the “trivial” core-image-minimal you can easily increase your build performance. I’ll admit that the time difference isn’t big in this comparison, but once you build more complex images the differences in the build times will become more apparent.

The cloud server used in this comparison costs about 15€ a month. If you want to give the cloud build a go, you can use this super-neat affiliate link to sign up and get 20€ worth of free credits. And maybe give me 10€ in the process if you happen to spend some actual money on the platform.

However, if you don’t have enough dollars for about two avocado toasts or three lattes every month, WSL is a better alternative than VMWare for building Yocto. Virtualbox seemed to be even worse than VMWare as it didn’t even want to build anything for me. It just spat out constant disk I/O errors that I simply couldn’t get fixed. Different caching options, physical disk drives, or even guest operating systems didn’t fix that.

But this is not all! You thought that you don’t have to read any more of these tables, didn’t you? I did some generic performance tests to get a bit better understanding of the different build machines, so I’ll add their results here as a bit of a bonus. The first one is the network speed test. I used the Ookla’s Speedtest command line tool for these. The values are averages of “about a dozen” runs.

There’s a bit of a difference in the latency between WSL and virtual machine, but the differences in the download and upload speeds are marginal. Cloud build is something entirely different than the other two, as expected.

Next is the hard drive speed, both reading and writing. For this, I used dd command-line tool. For the write test I used this command:

dd if=/dev/zero of=test bs=1G count=4 oflag=dsync

For the read test I used this command:

dd if=./test of=/dev/null

For the un-cached timings I cleared the caches with this command between the runs:

echo 3 | sudo tee /proc/sys/vm/drop_caches

For the cached timings I didn’t. Here are the results:

This is where the cloud machine’s SSD seems to shine, even if it didn’t do so well when doing the actual build work. The differences between WSL and VMWare on the other hand are a bit varied. What these results seem to suggest is that the uncached write performance is quite important for Yocto build performance. However, results also seem to be showing that the mass storage performance isn’t quite as important for the Yocto builds.

Finally, the CPU speed test. This was done with sysbench, using the command sysbench cpu --threads=4 run. Here I compared the CPU events per second:

All things considered, it’s slightly surprising how small the cloud build’s margin of victory when comparing the build times actually was. Especially considering how much more performant the CPU is according to the raw numbers. The most important thing actually seemed to be the number of cores, because a three-core cloud machine would already be slower for Yocto builds than a four-core VMWare virtual machine. So please remember this when you’re setting up your Hetzner cloud instance after you’ve registered through this link (I promise this affiliate pushing won’t become a thing).

That’s all for now. This is also most likely the last text about Yocto builds with WSL for time being. Next time it’s either something different that’s not related to Yocto at all, or then it’s something similar that is related to Yocto but not WSL.

tl;dr: WSL is faster than a virtual machine, but I recommend using cloud servers if possible

Writer of this blog post is wondering how he ever managed to finish his thesis