Esa Jaaskela - Embedded Software Handyman

How to build Fritzing for Windows

Fritzing is an open-source tool used to design and draw electrical wiring circuit diagrams, like this:

It can also be used to draw schematics and PCB diagrams, so it’s a really handy program indeed.

Why am I covering an open-source tool’s build process here? Because it was a surprisingly difficult process. You can buy the prebuilt version for 8€ and support the developer, but if you’re an adventurous soul (or a cheapskate) you of course want to build it yourself. I’m going to say that it took me almost a full afternoon to figure this build out, so the hourly rate wasn’t really that good, and the software is so useful that I’ll most likely be paying that 8€ anyway.

Here you can find the official Fritzing build instructions wiki. It’s a bit confusing at some points, but I’ll mostly follow the steps there, and mention it when I don’t.

This guide assumes you have a Windows machine with git, Visual Studio 2019 and a “sufficiently new” CMake installed.

Lets Go Sport GIF by ALL ELITE WRESTLING - Find & Share on GIPHY

Installing (correct) Qt version

If you have not installed Qt already, this step is quite straightforward. Download the Qt installer from here, agree with their open source policy and install the 5.15.2 version of Qt.

ERRATA: I did some more digging after publishing this text and realized that there actually is a simpler way to install multiple Qt versions than described in the next chapters. In the root-folder of the Qt installation there is tool named MaintenanceTool.exe. This tool can be used to install, remove and update versions of Qt. So use it instead of following these instructions. However, I’ll leave the chapters here as a proof that not everybody on the Internet is smart.

However, if you’ve already installed a different version of Qt this step was a bit tricky to complete (or at least I tried to make it difficult for me). Qt Creator (=Qt’s IDE) itself doesn’t allow downloading different versions of Qt and the official documentation only says that if you want to add a new version of Qt to Qt Creator you need to locate a qmake file. But where is this qmake file?

In the end, I couldn’t quite find a satisfactory answer to this. What I did was that I used the same installer as with the clean build, and installed the desired version of Qt in a different location. However, the installer forces you to install Qt Creator again, so it pollutes your system a bit.

If you’re not seeing the version you’d like to see, try checking Archive & LTS boxes from the menu on the right

After the version is installed, it can be linked in Qt Creator using this guide. Basically, just navigate to Edit->Preferences->Kits->Qt Versions->Add in Qt Creator and add the mythical qmake executable there (the executable is in a path something along these lines: Qt\5.15.2\msvc2019_64\bin\qmake.exe)

There is a small possibility that I’m just dumb and there is an “Install Qt version” button somewhere in the depths of the Qt Creator and I just couldn’t find it. But this approach at least works, even though it installs a bit of extra to the system. I also found the source packages for different versions, but didn’t feel like compiling Qt just to get Fritzing up and running.

Downloading the sources

There are a few repos that need to be pulled for the build. The first one is obviously Fritzing app itself. Besides that boost version 1.x.0 and libgit2 version 0.28.x are needed for building. For running the application you’ll also want Fritzing parts repo to get some actual components for your diagrams. All these repositories should be placed side-by-side so that you’ll end up with something like this:

Versions I used were:

fritzing-app: f0af53a9077f7cdecef31d231b85d8307de415d4
fritzing-parts: 4713511c894cb2894eae505b9307c6555afcc32c
libgit2: v0.28.5
boost: 1.79.0

Compiling dependencies

Next step is to compile libgit2. This is where I hit a big problem. Fritzing Wiki instructs to build with -DBUILD_SHARED_LIBS=OFF. However, with this flag, the build actually doesn’t output the .dll file required later on in the build. So the commands I used to build libgit2 actually were:

cd libgit2
mkdir build64
cd build64
#Note that BUILD_SHARED_LIBS is ON
cmake .. -G "Visual Studio 16 2019" -A x64 -DBUILD_SHARED_LIBS=ON -DBUILD_CLAR=OFF
cmake --build . --config Release

Boost and Fritzing parts shouldn’t require any compilation at this stage.

Compiling Fritzing

Next, we’ll get to open the Qt Creator, and open the phoenix.pro file located in the root of the fritzing-app folder. Configure it for the 5.15.2 version of the Qt, and as the build wiki instructs, add the following to the Projects->Run->Command Line Arguments in Qt Creator:

-f "/path/to/fritzing-app/" -parts "/path/to/fritzing-parts/" -db "/path/to/fritzing-parts/parts.db"

After this is done there’s still one more hurdle to overcome. Building now seems to result in this error:

error: dependent 'F:\Esa\Documents\Fritzing\debug64\ui_fabuploaddialog.h' does not exist.

Detective Looking GIF by Sherlock Holmes Games - Find & Share on GIPHY

To fix this issue, we’ll need to navigate to the build folder that gets generated alongside the fritzing-app folder, and is named like build-phoenix-* (rest of it depends a bit on your build configuration). There we need to use Qt’s jom to build compiler_uic_make_all target:

P:\ath_to_QT\Tools\QtCreator\bin\jom\jom.exe -f Makefile.Debug compiler_uic_make_all

This will generate the missing headers to the debug64-folder that’s also alongside the fritzing-app and build folders. After this, the build should be as simple as clicking the green arrow in Qt Creator. If you get a boost-include error, make sure you have the boost folder directly under the boost_1_79_0 folder, and that you don’t have a structure boost_1_79_0\boost_1_79_0\boost as I did. This wrong structure resulted in the following error:

F:\Esa\Documents\Fritzing\fritzing-app\src\svg\groundplanegenerator.cpp:40: error: C1083: Cannot open include file: 'boost/math/special_functions/relative_difference.hpp': No such file or directory

Once the build completes, the Fritzing will start. Because there is db-argument given in the command line, the actual program won’t start. Instead, Fritzing generates the parts database and closes itself after the process finishes.

Running Fritzing

Congratulations, you should now have built & prepared Fritzing! After the initial build & database generation remove the -db argument from the run arguments so that Fritzing starts properly with Qt Creator. This type of launch was a good enough solution for me, and I didn’t feel like going through the hassle of creating an actual executable for Fritzing. I think I can pay 8€ for that pleasure.

Stay tuned for texts that include more Fritzing diagrams!

July 27, 2022 by ejaaskel Random ramblings 0

Yocto and WSL, part 3: WSL vs. VMWare

Read the previous parts of “Yocto and WSL” series:
Yocto? On WSL2? Easier than you think!
Yocto and WSL, part 2: The Graphic Boogaloo

The ultimate showdown of the ultimate destiny!

All the good things are trilogies. Star Wars original trilogy, Nolan’s Batman trilogy, and The Hobbit trilogy. And now, Yocto & WSL trilogy. As per usual with the trilogies, the last part may be a bit of a letdown for the hardcore fans. This time we’re not doing really anything technical or exciting. Instead, we’re comparing some numbers while trying to decide if this whole exercise was worth it or not.

To judge the usefulness of WSL in the Yocto build context were doing a comparison between WSL machine and a “regular” virtual machine to see which one is actually better for building Yocto (if you’re forced to use Windows). Better means faster in this case. For this purpose, I created a VMWare virtual machine that matched my WSL machine’s specs: 4 cores, 8GB, and enough disk space to accommodate the build area. For the operating system I chose Ubuntu Server 20.04.4, and as a reference build target I used the simplest core-image-minimal recipe with both machines.

As a bit of an extra, I’ll be adding a comparison to a cloud build server as well for the same price! For this purpose, I prepared a similar 4-core 8GB build server on Hetzner. This is a virtual machine as well, but it’s interesting to have more things to compare to. Especially because the cloud instances provide an option to scale up in case they aren’t powerful enough. Also, like WSL and VMWare, they can be used as an alternative for a Linux build machine. Without further ado, let’s get into crunching numbers.

First, to actually make a worthwhile comparison between different build times, I’m going to separate the process of downloading the sources from the actual build itself. I did three runs of downloads because “third time’s the charm”. VMWare was using a NAT network connection here. The separation was done because the time it takes to download the sources depends on the load on the source code servers, how much my neighbor happens to be using the internet at the given moment, and the alignment of the stars. Or what do you think of these results:

	Mean	Median	Run 1	Run 2	Run 3
WSL	44m 4s	52m 1s	52m 1s	59m 29s	20m 42s
VMWare	46m 20s	45m 5s	37m 28s	56m 29s	45m 5s
Cloud build	35m 18s	33m 40s	33m 40s	41m 49s	30m 26s

There were two packages that became bottlenecks in these tests: cross-localedef-native and linux-yocto. Especially the former. Other packages were downloaded in a few minutes but cross-localedef-native took almost always at least half an hour to download. In the WSL run #3 cross-localedef-native took only about 15 minutes to download, and instantly the whole download process was a lot quicker. In general, I would still boldly claim that Hetzner build is potentially the fastest on average. Or at least it would make sense, as the Internet pipes in their data centers are most likely wider than mine at my home.

That good ol’ YouTube player brings a nostalgic tear to my eye

Then, the actual build times. For these, I actually did four runs, because “third time’s the charm and one more for a good measure” (this blog text isn’t quite as scientific as I make it out to be). As mentioned earlier, I built the core-image-minimal for all these runs. These builds were done without downloading the sources in an attempt to get the build times to be a bit more stable. Here’s the table:

	Mean	Median	Run 1	Run 2	Run 3	Run 4
WSL	109m 15s	106m 48s	106m 11s	121m 26s	107m 26s	103m 37s
VMWare	125m 51s	125m 51s	126m 33s	126m 32s	125m 10s	125m 9s
Cloud build	106m 30s	105m 35s	111m 10s	105m 29s	105m 3s	105m 40s

As perhaps expected, the virtual machine was constantly the slowest option. What however is slightly surprising is that the WSL was almost as fast as the cloud build server with the same resources. Or actually, with better resources as the build server had an SSD disk while WSL was running on an old HDD I bought from a friend of mine for 30 euros some years ago.

But yeah, in the light of this evidence I’d say that if you’ve got some dollars to spare, getting a cloud build server is the best option for building Yocto (if you can’t get a native Linux build machine that is). It’s not only the fastest option, but it’s also scalable, so when you’re building something else than the “trivial” core-image-minimal you can easily increase your build performance. I’ll admit that the time difference isn’t big in this comparison, but once you build more complex images the differences in the build times will become more apparent.

The cloud server used in this comparison costs about 15€ a month. If you want to give the cloud build a go, you can use this super-neat affiliate link to sign up and get 20€ worth of free credits. And maybe give me 10€ in the process if you happen to spend some actual money on the platform.

Shooting Easy Money GIF by iHeartRadio San Francisco - Find & Share on GIPHY — How well I handle my money every time the paycheck comes in

However, if you don’t have enough dollars for about two avocado toasts or three lattes every month, WSL is a better alternative than VMWare for building Yocto. Virtualbox seemed to be even worse than VMWare as it didn’t even want to build anything for me. It just spat out constant disk I/O errors that I simply couldn’t get fixed. Different caching options, physical disk drives, or even guest operating systems didn’t fix that.

But this is not all! You thought that you don’t have to read any more of these tables, didn’t you? I did some generic performance tests to get a bit better understanding of the different build machines, so I’ll add their results here as a bit of a bonus. The first one is the network speed test. I used the Ookla’s Speedtest command line tool for these. The values are averages of “about a dozen” runs.

	Latency	Download	Upload
WSL	3.76ms	104.22 Mb/s	59.85 Mb/s
VMWare	3.94ms	104.13 Mb/s	59.88 Mb/s
Cloud build	0ms	9266 Mb/s	9220 Mb/s

There’s a bit of a difference in the latency between WSL and virtual machine, but the differences in the download and upload speeds are marginal. Cloud build is something entirely different than the other two, as expected.

Next is the hard drive speed, both reading and writing. For this, I used dd command-line tool. For the write test I used this command:

dd if=/dev/zero of=test bs=1G count=4 oflag=dsync

For the read test I used this command:

dd if=./test of=/dev/null

For the un-cached timings I cleared the caches with this command between the runs:

echo 3 | sudo tee /proc/sys/vm/drop_caches

For the cached timings I didn’t. Here are the results:

	Write	Write (cached)	Read	Read (cached)
WSL	91.6 MB/s	93.7 MB/s	91.9 MB/s	628 MB/s
VMWare	50.6 MB/s	107.5 MB/s	338 MB/s	559 MB/s
Cloud build	1.2 GB/s	1.3 GB/s	603 MB/s	1.2 GB/s

This is where the cloud machine’s SSD seems to shine, even if it didn’t do so well when doing the actual build work. The differences between WSL and VMWare on the other hand are a bit varied. What these results seem to suggest is that the uncached write performance is quite important for Yocto build performance. However, results also seem to be showing that the mass storage performance isn’t quite as important for the Yocto builds.

Finally, the CPU speed test. This was done with sysbench, using the command sysbench cpu --threads=4 run. Here I compared the CPU events per second:

	CPU events per second
WSL	5359.31
VMWare	5284.58
Cloud build	14582.81

All things considered, it’s slightly surprising how small the cloud build’s margin of victory when comparing the build times actually was. Especially considering how much more performant the CPU is according to the raw numbers. The most important thing actually seemed to be the number of cores, because a three-core cloud machine would already be slower for Yocto builds than a four-core VMWare virtual machine. So please remember this when you’re setting up your Hetzner cloud instance after you’ve registered through this link (I promise this affiliate pushing won’t become a thing).

That’s all for now. This is also most likely the last text about Yocto builds with WSL for time being. Next time it’s either something different that’s not related to Yocto at all, or then it’s something similar that is related to Yocto but not WSL.

tl;dr: WSL is faster than a virtual machine, but I recommend using cloud servers if possible

Writer of this blog post is wondering how he ever managed to finish his thesis

June 15, 2022 by ejaaskel Yocto 0

Yocto and WSL, part 2: The Graphic Boogaloo

Read the other parts of “Yocto and WSL” series:
Yocto? On WSL2? Easier than you think!
Yocto and WSL, part 3: WSL vs. VMWare

So, in the previous part, we got the Yocto built with WSL and tested it out with a text-based terminal interface. This time we’re going to improve things a bit by getting the graphics working! Like every engineer always says, “it’s really nice to have this fancy graphical interface with buttons that have rounded corners instead of an efficient, text-based interface”.

(After publishing this text it was brought to my attention that WSL on Windows 11 actually supports GUI applications natively. I’m not sure how this works with QEMU but I’d recommend trying that out before going through all the hassle mentioned in this text. If you’re still stuck on Windows 10 like me these steps are at least currently applicable)

The process of getting graphics up and running in the WSL consists of three steps:

Make a hole into the Windows firewall (starts promisingly) so that WSL can communicate with an X server
Launch an X server with the access control disabled (this sounds great as well)
Set the environment variables in WSL

The instructions I used to achieve this can be found in this great blog post. However, for the sake of documentation and out of the fear of broken hyperlinks, I’ll go through the steps here as well.

The hole in the firewall needs to be made because the networking in WSL2 is implemented in a way where the traffic is considered to be coming from an external source (as opposed to WSL1). The firewall rule can be set using the following steps:

Search “Windows Defender Firewall with Advanced Security” from the Start Menu
Click “Inbound Rules” on the left-hand side menu, and then click “New Rule…” on the right-hand side menu
Select rule type to be “Port”, set the port to 6000 and give it a good name. The rest of the settings should be good by default (TCP protocol & allowing all kinds of access)
Right click on the newly created rule in the rule list, select Properties and from there select Scope-tab. The limited scope prevents unwanted entities from exploiting the poor firewall’s new rule. Check the image below for Scope-settings:

Basically, we’re limiting the connections just to private IP-addresses

And just like that, the firewall should allow the traffic from WSL2 to the X server. The X server that we haven’t even installed yet. I have been using VcXsrv X server, and the next step is downloading and installing it. After VcXsrv has been installed we can continue.

As briefly mentioned earlier, the access control needs to be disabled when launching the server to make things go smoothly. Just use -ac flag when launching the X server from the command line to disable access control. The full command to launch VcXsrv from a Windows terminal is:

vcxsrv.exe :0 -multiwindow -wgl -ac -silent-dup-error

The first parameter sets the display number to zero, the second one allows multiple windows, the third one doesn’t seem to be necessary after a quick test but I’ll still leave it here because it was in the original post, the fourth parameter is the important one that disables the access control and the last one prevents errors if launching multiple VcXsrv-servers. After getting the server running we’re almost ready to get into the exciting world of graphic content. Open up a WSL session, and export the following two variables (in WSL terminal):

export DISPLAY=$(awk '/nameserver / {print $2; exit}' /etc/resolv.conf 2>/dev/null):0
export LIBGL_ALWAYS_INDIRECT=1

The first variable sets the display we want to use (basically WSL nameserver address and display 0) and the second variable should prevent skipping X server when calling draw commands. I’m not 100% what that means, but it sounds important so it’s better to set it.

And now, finally, we’re ready to check out the graphical interface! If you’ve closed your WSL session in the past two months it took me to write this second blog post, first head to your poky-folder and run source oe-init-build-env again. Then you should be able to run:

runqemu
#look, no no-graphic option this time, wowzers!

*Wait, I was promised graphics, what is this? I want to click icons and drag-n-drop stuff around.*

Well, the thing is that the core-image-minimal image we built in the previous blog post doesn’t have a graphical interface built. While technically this new window is a graphical, emulated window, it’s not a graphical user interface. To get an image with these capabilities, we need to build core-image-sato instead (sorry).

So yeah, navigate to your build-folder and run bitbake core-image-sato. Even though the build reuses some of the artifacts from the previous build this will take some time, so be prepared to find something else to do for a while. After the build has been completed, you should be able to run runqemu again, and see something like this:

The apparent problem is the quite significant latency introduced by QEMU, and running QEMU on top of the WSL doesn’t do the situation any favors. Technically you can run the image also on a Windows native QEMU, but I didn’t see any noticeable difference in the GUI performance. My method of analysis was not the most scientific one though: I just pressed a button and tried to internally analyze how frustrated I got before something happened in the GUI. With both WSL QEMU and Windows native QEMU I got “quite frustrated”, so I’d call it a tie.

I didn’t do any actual benchmarking on if there’s a difference in the computing performance, perhaps in some upcoming blog post I could write a bit about that. It seems like the Windows native QEMU lacks some options (I couldn’t get the networking working (more like notworking amirite)), so it may be better to stick to WSL QEMU.

That’s all for this time. Hopefully, you learned something from this post, I at least did. Once again big shout-out to Skeptric-blog, plenty of other interesting texts in there as well.

Part 3 is now available here!

PS: I disabled the comments on this blog. It seems like the only ones writing those were freaky spambots. If you’ve got something to ask or say you can reach me on LinkedIn.

March 1, 2022 by ejaaskel Yocto 0

Yocto? On WSL2? Easier than you think!

Read the other parts of “Yocto and WSL” series:
Yocto and WSL, part 2: The Graphic Boogaloo
Yocto and WSL, part 3: WSL vs. VMWare

(Sorry for the clickbaity title, but the WordPress Headline Analyzer gave it the best score I’ve ever seen so I can’t change it)

Let’s begin this post with the basics. What are these words and acronyms in the title? Some nerd stuff? Well, yes. Yocto is a project/tool that can be used to generate a cross-compilation toolchain and a firmware image for an embedded system. This sort of work includes never-ending woes with bootloaders, Linux, and much, MUCH more. Windows Subsystem for Linux (= WSL) is “a compatibility layer for running Linux inside Windows”. It’s not really a virtual machine (at least according to Microsoft) or a container, despite running a virtual machine and virtualization being its core (or so I understood). What is it then exactly?

Well, let’s call it a virtualization layer and settle for a fact that it’s a way of running a Linux-like environment inside Windows with relative ease. And as you can guess from the Yocto project’s description, we’re going to be needing plenty of that Linux.

First things first: setting up WSL2. There are plenty of tutorials and official documentation on this topic, so I’m not going too much in-depth with this one (although I’ll go through the issues I faced along the way). I recommend checking at least this official documentation out (There actually seems to be an easier way of installing WSL these days, but I haven’t tried it). In a nutshell, the steps that should be done are:

Enable WSL feature in Windows (and reboot)
Install WSL2 kernel update (most likely reboot after this one won’t hurt either)
Set WSL2 as the default WSL version (reboot isn’t really needed here, but it’s always fun to stare at that picture of a cave that’s the log-in screen background and think how you could be somewhere else. Somewhere warmer where the sun shines)
Install the desired distribution.

It’s Ubuntu that you desire. Or at least I recommend it because it’s the most widely supported Linux distro [citation needed] and the Yocto guide explains the steps a bit more specifically for Ubuntu users. Other distros still work, or at least should work. Yocto claims to support “recent releases of Fedora, openSUSE, CentOS, Debian, or Ubuntu”, whatever that means.

However, this installation phase was where I started to face problems. When installing Ubuntu I got the following error:

Installing, this may take a few minutes…
WslRegisterDistribution failed with error: 0x80004005
Error: 0x80004005 Unspecified error

Despite the ominous-sounding “Unspecified error” this was caused by the simple fact that I didn’t have enough disk space on the disk that contained AppData-folder (I guess this defaults to C:). There seem to be some other explanations for this error on the Internet, but I recommend freeing space if you’ve filled your C: with junk (as I assume every Windows user has done). 2GB of free space wasn’t enough, 7GB seemed to do the trick.

After doing this and retrying, I got the following error instead:

WslRegisterDistribution failed with error: 0xc03a001a
Error: 0xc03a001a The requested operation could not be completed due to a virtual disk system limitation. Virtual hard disk files must be uncompressed and unencrypted and must not be sparse.

This one was a bit trickier, but I found the fix here. Basically, you need to navigate to %LOCALAPPDATA%/packages/, check the (advanced) properties of CanonicalGroupLimited.UbuntuonWindows-folder and ensure that it isn’t compressed or encrypted.

What, an actually useful image in this blog? Didn’t see this one coming. Fortunately, my bad paintwriting makes it feel a bit less useful.

And after these trials, the Ubuntu image should finally install without further problems. Or at least it did for me, YMMV. Once you get the WSL up and running, I recommend playing around with it a bit if it’s not familiar to you yet. If you don’t know where to start, this video can be useful (About 3:40 onwards). To be honest, so far the biggest issue I’ve had with WSL is how simple it is, I was expecting it to be a lot more complex and cumbersome.

After getting the basics of WSL down it’s time to get started with Yocto. Well, almost. As you may guess, the Yocto project takes up disk space. A lot it. The official requirement at the time of writing is 50GB. However, as you may remember from the earlier chapters, I have barely 7GB free on the disk where the WSL image gets installed by default.

A bit of googling revealed two useful options to the Windows’ WSL command-line tool: export and import. These can be used to pack the image and then load it in a non-default location. unlink option could be used to get rid of the obsolete machine. So in short, the Powershell commands to be run on Windows are:

wsl --export Ubuntu E:\yoctotest\testDistro.tar
wsl --unlink Ubuntu
wsl --import UbuntuCustom E:\yoctotest\UbuntuCustom E:\yoctotest\testDistro.tar
#Or something similar, depends a bit on what you actually want and what drives you have

As you may have noticed at this point, all of my problems were caused by the lack of disk space on C: drive, so if you don’t have that problem I guess your experience will be a lot smoother.

Just to prove to you that I have learned absolutely nothing from this “out of disk space”-experience, here is a 1.5MB gif.

Finally, after getting the WSL image ready and in the correct location, it is time to properly start working on the Linux side of things. There’s one more thing to do before moving to Yocto: updating apt and installing zstd. I’m not sure why zstd isn’t listed in Yocto requirements, but my build failed without it. These commands can be run in the Ubuntu shell to get the things ready for Yocto:

sudo apt update
sudo apt install zstd

After that, it’s mostly just a matter of following the Yocto quick start, with the difference of building the core-image-minimal instead of core-image-sato. Sato-image is the GUI-image, but we don’t have graphics yet so it’s better to stick to core-image-minimal. In a nutshell, the steps for Yocto-build are:

Install dependencies (no need to reboot)
Clone poky-repository and check out the latest LTS (=long-term support) branch dunfell (interestingly, no reboot)
Source the build environment (don’t reboot, you’ll lose the sourced environment)
Build the core-image-minimal. Did I mention core-image-minimal yet?

Yocto-build with WSL will print a warning that you better be sure that you have enough disk space, but besides that it should work normally. With the default settings, the build will eat pretty much all of the resources your machine has (it’s been optimized that way), so don’t be afraid if things start to freeze a bit. Especially memory tends to be a bottleneck, partially because of WSL. However, I was able to watch a Youtube video most of the time while waiting for the thing to finish, so I guess it doesn’t render the computer completely useless. During the processing of the heavier components it may be better to do something with a video game console of your choice though.

And if the stars are aligned, the build will complete in an hour or two (or six)! The dunfell-branch is usually quite stable (as LTS branches should be), so the build shouldn’t be in a broken state. After the compilation is done, you can try out the generated image with the following command:

runqemu -nographic

runqemu command is Yocto’s wrapper for QEMU, which is an emulator that can be used to run the built images (and plenty of other things outside Yocto as well). runqemu helps to automatically find and set up the image for the emulator. -nographic option prevents loading a graphical shell in a separate window, as the graphics are not set up for WSL yet.

And that’s it! It is easy once you know how to do it. Like pretty much everything else I guess. But I was pleasantly surprised that Yocto officially supports building in WSL, so there was no need for (dirty) hacks. The memory seems to be a bit of a problem, but that’s always the case. In the next post there’ll hopefully be some graphics!

Part 2 is now available here!

December 16, 2021 by ejaaskel Yocto 0

Delayyyyyy: v0.1.0, The First Release

Time for a celebration! The 0.1.0 release is here. It took quite a lot of effort to get everything in place and tested. As usual, the last mile turned out to be more difficult than expected. But in the end, things finally felt good enough for the first beta release. I broke the philosophy of having only one logical change in a single pull request by stuffing all the final changes into one PR, sorry about that. In this blog post, I’m going to go briefly through those commits that lead up to the beta release. The section headers are links to individual commits in case you want to judge my code.

Fix bug in short BPM synced delays

So, first things first, the cliffhanger I mentioned in the previous blog post: the bug in the synced delays. The bug was that when there is a high amount of echos, but not enough synced time signature values for them, the plugin went quiet. Instead, there should have been the few short echoes that were available, but still less than the specified amount. This was caused by prematurely breaking out of the loop that creates the delay buffers. Breaking out of the loop would have been fine if we would have been incrementing the iterator (i.e. creating the longest delay first). However, the loop iterates in reverse order (i.e. creating shortest delay first) and thus we can’t break from the loop before going through all synced time intervals.

It may not be a good idea to lie to the user about the final number of echos though.

Fix parameter state storage

I finally tried a simple saving of parameters in a digital audio workstation project. As it turns out, empty functions don’t do much of saving (or loading), and when I reloaded the project the carefully set parameters were gone. The solution was simple, actually it only consisted of copy-pasting code from an official JUCE tutorial. Not only for the saving but the loading as well. One feature isn’t much of use without the other.

Get BPM from a possible VST host

This is the big one. Basically, I did the following things to achieve this:

Run setDelayBufferParams in it’s own thread
Read BPM (beats per minute) from the playhead object
Detect if BPM changes

The first change was done out of necessity to make the second change possible. Playhead is an object that contains audio playback information, like the position of the audio track or the BPM (if available). However, this object can only be read in the processBlock-function. I can’t remember if I’ve mentioned this earlier, but processBlock-function should execute as quickly as possible to keep audio latency as small as possible.

So, we can’t run setDelayBufferParams in the processBlock because we want to keep things smooth. Instead, a separate thread needs to be created and notified in case the BPM (or any other parameter) changes. The thread also checks parameters every now and then by itself. For thread control, I tried to make a sort of one-sided locking where setDelayBufferParams always waits for the processBlock to finish (even if it’s not running, because it’ll run soon anyway), and then hopefully run before the next block has to be processed because the processBlock is a really busy function and it has no time to wait for anything or anyone.

However, thread-control based assumptions that things “should definitely work this way” will fail with 100% certainty. This is no exception. I’ll fix this in a future commit, but please keep this silly design in mind (and then forget it so you won’t create it yourself)

Add plugin header and version info

What’s worse than getting a bug report? Getting a bug report without the version information, or even the name of the product. This commit added a neat little header that shows the name of the plugin, the name of the creator, and even the version in the bottom right corner. I should have dropped the name of the creator though, so no one could send me bug reports.

This commit uses a neat UI feature of JUCE: removeFrom. This can be used to crop (and return) a certain section of the screen area. Something like this is perfect for the headers. Just remove a bit from the top, and the removed area is the new header area! No need to break the existing flexbox-based UI to try and fit header info in there.

Remove negativity, add Fonzie and update TODO

Does what it says, all changes here went to the README. There may be better issue management tools than just updating the README, but at least it’s not as bad as JIRA.

I have watched zero episodes of Happy Days in my life, hopefully I’m using this correctly.

Set the initial values of parameters to default values

This commit sets the initial values of the parameters to their default values. Would you have guessed? Basically makes the code a bit neater. However, this introduced a quite big bug that I somehow missed: the plugin makes no sound before the UI elements are edited because the delay buffers only get created if the parameter values change. If the parameter values are initialized to their default values, the buffers won’t get created before UI changes to something non-default. This programming bug is equivalent to a fist-sized cockroach in real life. I reverted this change here shortly after merging it, but I may need to revisit the idea some other day.

And then I thought that I was ready for the 0.1.0 release. I did some final testing of the plug-in, fiddling around and making sounds. Life was beautiful. Until suddenly, a deafening “pop”-sound occurred. Ableton recorded it to be 173 dBFS. Fortunately, I wasn’t using my headphones, otherwise my head would have exploded. People from miles away heard the sound of my mistake, which was slightly embarrassing.

Fun fact: the loudest sound on earth has most likely originated from the Krakatoa volcano eruption in 1883. It was recorded to be 172 decibels 100 miles away. And that’s probably the most educational thing in this blog so far.

Well, the reality is a bit less exciting to be honest. The sound wasn’t that loud, and I guess Ableton just calculated it “a bit” wrong. But there was a pop, and it was reproducible easily: just by wiggling the controls of the plugin for a minute or two.

And this is where my “slightly” “stupid” approach to thread control comes in.

Fix race-condition in setDelayBufferParams and processBlock

Basically, it’s dumb dumb very very dumb to assume that some things can and will always run in a certain time frame. In the end, it’s better to have a bit more latency than weird popping audio artifacts. Basically, the sound occurred when setDelayBufferParams didn’t finish before the processBlock was called. As you remember, processBlock waits for no one. So setDelayBufferParams was happily replacing/removing the delay buffers and processBlock was happily reading those garbled memory locations, and a pop sound occurred. That’s why I added a wait to the processBlock so that it won’t run too early. Not the best possible design, but it’ll have to do for now (and most likely until the end).

Add companyName for VST information

Finally, while testing out the plug-in I noticed that the company name wasn’t set correctly (it said yourcompany or something like that), so I fixed it to point to my musical alter ego.

And that’s it! That’s the story of how 0.1.0 came to be. I made one final PR where I added the ready-built x64 VST3-file and updated the README with the known issues of the plugin being a CPU hog and popping occurring while editing parameters. The CPU optimization is worth another story, but I already have a few ideas for that (and for the popping as well). But meanwhile, you can download the VST3 plugin here:

Delayyyyyy_0.1.0 release

This is how it feels to optimize. Even the quality of the GIF is optimized.

November 2, 2021 by ejaaskel VST Plugins 0

Delayyyyyy: BPM synced delay effect

As a hobby project, I have been working on a VST plugin lately. For people who are unaware of what a VST plug-in is (I assume this is most of the people), it’s basically an audio effect or instrument used with DAWs (digital audio workstations). To simplify the concept even further, it’s a piece of software that either creates or modifies an audio or MIDI signal.

The plugin I’ve been working on for the past few months is an audio delay/echo plugin. It takes an audio signal and repeats it a given amount of times with some certain maximum delay value. It also has few simple options: decay time for deciding how quickly the repeated signal levels decay, and a “ping pong” to alternate the left and right sides of the signal.

Ta-dah! This how the plug-in looks at the moment.

Now that I’ve briefly explained what the project is about, I can talk about my latest addition to the plugin: BPM synced delay (basically a delay that is synced to the rhythm of the music). I’ll admit, this may feel a bit like jumping into a moving train when there’s existing code that hasn’t been explained in previous blog posts, but I’ll try my best to keep this simple & understandable. Here’s a link to the PR that contains the BPM sync feature I’ll be talking about in this blog post.

There are changes to two classes, PluginEditor and PluginProcessor. The former is responsible for the UI, and the latter for the actual signal processing.

In the PluginEditor changes were quite simple. I added a BPM sync check box that toggles the visibility of the synced and unsynced delay sliders. I actually had most of the stuff already in place, just commented out (commenting out code is definitely the best method of version control, trust me on this, I’ve had “senior” in my title for three whole months /s). Checkbox’ value is also used in the PluginProcessor side to see if the delay should be synced or not. I’m using CustomSliders to have a custom look and feel on the sliders, but everything should work on a regular slider class equally well (or even better).

There are also some UI gymnastics going on to get everything aligned correctly. The UI should be resizable (kinda), which causes its own headaches there. Possibly the most interesting thing is the custom textFromValueFunction that allows showing musical time signatures instead of raw slider values in the slider value box. Besides these, I also had to inherit the Button listener to be able to update UI and delay parameters whenever the user would click the button. There are more elegant ways to do this, but I think for two lines of code in a hobby project this is a sufficient solution.

Simple so far? Well, things should get a bit more interesting (i.e. complicated) in PluginProcessor. First of all, I had to create parameters for the synced delay toggle and synced delay amount to ValueTree. ValueTree is a class that, you guessed it, stores different values and helps handling them.

However, maybe the most important functional changes in the PR are to the setDelayBufferParams-function, and that could definitely use some explanation. The delay in this plugin works by creating a vector of size N, where N is the number of echos that we hear back. This vector is then filled with ring buffers, and these buffers have read and write pointers that are offset by a certain amount from each other. There’s a picture after this, and I hope that it’ll explain the idea better than two chapters of words and poorly thought-out mathematical formulas.

The offset between read and write is calculated from the maximum delay time X and echo amount Y user set from the UI. The first buffer gets a value of X/(2(Y-1)), second X/(2(Y-2)), and so on, until (2(Y-n)=0. In that situation, the last buffer just gets the value X. This creates an interesting (yeah for sure) non-linear, yet fairly symmetrical delay effect. Please note that the first echo buffer has the smallest offset between read and write pointers.

ANYWAY, after this offset is calculated, the write pointer is set to this value while the read pointer is set to zero. When there’s an audio signal coming in, there’s first a read performed on the buffer to obtain the possible previous, delayed signal. Then the incoming signal with possible ping pong and decay applied is written to the buffer. Finally, the earlier read value is added to the incoming signal. This is then repeated for all the buffers. Since each buffer has a different, bigger than zero offset between write and read pointers, the audio signal gets repeated at different intervals, creating a delay effect. Wowzers!

In this example, the maximum delay is set to one second, and there are four echoes. I’ve been told to never apologize using Paint in explaining things, but hopefully this makes sense.

Now, how does this idea work in a rhythmically synced world? Well, time for a little music theory for a change. 97% of the modern western music is in 4/4 time signature, meaning that there are four kick drum hits in a bar, and music is constructed of building blocks with a size of four bars (gross oversimplification, but I don’t want to end up neck-deep in music theory because I’ll explain it wrong). So in a track with a BPM of 120 there are 120 kick drum hits in a minute, and 30 bars in a minute.

This gets a bit subjective from here, but I personally like synced delays to be divisions and multiples of two from the previously calculated value, starting from 1 bar, to create a nice rhythm that sounds good on most music. This means that the delay gets values like 1/2, 1/4, 1/8, and 1/16 when going shorter than one bar, and values like 2, 4, 8 & 16 bars when lasting longer than one bar. When we know the beats-per-minute, or BPM, it becomes fairly trivial to calculate the write and read pointer offsets that are rhythmically synced. Instead of using the user set value for the maximum delay, we calculate the time for a certain amount of beats in a certain BPM.

Looks similar, doesn’t it? Max delay set to 1 bar, and echoes to 4. The difference is that the duration of the delay doesn’t depend on the time, it depends on the tempo of the music

Hopefully, you’ve understood what I’ve tried to say with this. To make this code change slightly more confusing, there are some optimization changes thrown in there as well. Basically, earlier I had a static 10-second size for the delay buffers. However, with synced delays, this may not be enough, and for the shorter un-synced delays, this will be way too much. So I set the buffer’s size to be equal to the write pointer offset because there is no point in having the buffer be longer than that. The ring buffer will rotate more often, but that isn’t really an issue because old values are not needed.

And that’s the process of how I did the BPM synced delays to my VST plugin. The theory is quite simple once you get the idea of how I implemented the delay buffers, after that it’s just a matter of modifying the write pointer offset. The rhythm is still hardcoded to 120 BPM, but it’ll be fixed in the next PR. There’s also one bug in this implementation that I’ve fixed on my local branch already, but I’ll leave details on that as a cliffhanger to my next blog post.

October 13, 2021 by ejaaskel VST Plugins 0

How to start a (tech) blog with WordPress & DigitalOcean

Haven’t we all dreamt about it? Writing some simple tutorial blogs for people who don’t want to go through man-pages and watching the ad revenue pour in while building the personal brand to ensure success in every aspect of life. The only problem: you don’t have that blog yet. Fortunately, there are both simple and not-so-simple solutions to this!

Please note that despite the clickbaity title this isn’t really a step-by-step tutorial, although it may be useful when starting out with the technologies mentioned in the title. It’s more of documentation of the things I did to get this site you’re reading now up and running.

Option A: The casual approach
Step 1: (well okay there are steps, but this still isn’t a tutorial)
Go to wordpress.com, create your free account and start writing.

This is definitely a valid approach, but it sounds easy, doesn’t it? A bit too easy one might say…

Option B: The tech approach
Step 1: Obtain the domain
Buy the domain you want. I personally use Namesilo as a domain registrar. It is cheap, and as a nice bonus, it often reminds you that you get what you pay for. There are other alternatives that could definitely be considered. GoDaddy at least seems to be commonly used, and often many services provide documentation on how to get their service working with GoDaddy.

Whatever registrar you choose, buy the domain you want to use. The process isn’t much harder than ordering anything else online. Often registrars try to sell all sorts of extras, you don’t really necessarily need any of those. If you like to wear hats made of tin foil like me then the WHOIS privacy is the most useful one if it’s offered.

Step 2: Obtain the server
Go to digitalocean.com and create an account. DigitalOcean is a cloud infrastructure provider, and what we want from them is a virtual machine instance that runs WordPress (and Apache, and MySQL, and PHP, and Linux, and…). Fortunately, we are not the first people who want to do this, and the process is really streamlined. DigitalOcean even talks about One-Click Install, although there definitely are more clicks than that. After you’ve created the account, go here and get started with the process:
https://marketplace.digitalocean.com/apps/wordpress

I recommend following DigitalOcean’s instructions because they most likely are going to be more up-to-date than mine. When there are questions about some extra services or premium CPUs or what-not, just say no. Your blog isn’t going to be a big hit in the first few days/months/years anyway, so no use wasting money on 12 CPUs yet. Backups might be useful though after you get the things rolling.

After you’ve clicked through the marketplace, you should be greeted with a screen where you can see your Droplet (DigitalOcean’s fancy name for a virtual machine). It’ll take a moment for the machine to initialize. When it’s done, you can click the Getting Started button in the middle of the screen to get started (these instructions will most likely be quite similar to the instructions in the marketplace link above):

The first step of the instructions at least at the time of writing was making an SSH connection to the Droplet. You can either use your favorite SSH client or DigitalOcean’s browser console. After an SSH connection was formed, an automatic helper script actually helped to finish the configuration (it seems that the SSH connection may close few times if you try it before the machine has properly settled). The setup is mostly basic stuff, setting up the admin account, etc, but the script also asks about LetsEncrypt. Skip it for now, as the DNS settings aren’t yet in place.

If you break something, fear not. You can trash the Droplet easily:

Then just create a new one and start from the beginning. I did this a few times.

Step 3: Obtain the DNS
After the Droplet has been created and WordPress has been installed, it’s time to set up the DNS. Now, I’m going to be honest here, I don’t know much about this stuff so that’s why I’m putting a lot of pictures in this section. Basically, there are two things that need to be done: changing the nameserver on your domain registrar to DigitalOcean’s nameservers and setting up the DNS in DigitalOcean. The first step varies a bit depending on which registrar you chose earlier, here’s how it is done in Namesilo:

Select this thingy right here (stack of pancakes?)

…and enter the DigitalOcean nameservers.

Setting up the DNS in DigitalOcean is fairly simple, the following steps should create correct DNS settings almost automatically:

…and finally this. Or something similar to this. I’m not 100% sure if the www A-record is required, but at least it doesn’t hurt. I guess. I tried getting this right more times than I want to admit, and when I finally got things working I decided that I don’t want to touch the settings ever again.

The changes done in these steps will take some time to actually have an effect. I’ve seen a correct domain name resolution happen in five minutes, and I’ve seen it not happen in few hours. I noticed that Firefox Focus is the most efficient browser in destroying its caches, so I’d recommend using it to verify that you’ve set up DNS correctly. The private mode of regular browsers doesn’t seem to help anything.

Step 4: Obtain the HTTPS
After you’ve set up the DNS properly, you can and definitely should set up the SSL with LetsEncrypt. This is actually quite straightforward. Just connect to the Droplet via SSH again and run the following command:

certbot --apache -d yourdomain.com -d www.yourdomain.com

I think you could actually set up the DNS before setting up the VM and create the certificates with the helper script during the initialization, but at least I did things the hard way.

Step 5: Obtain the stress of updating WordPress plugins and seeing the cash flow out of your pocket
At this point, most of the actually stressful work should be done. Unless you’re like me and already forgot your WordPress admin password. In case you have a better memory than I do, you should be able to navigate to yourdomain.com/wp-admin and log in to WordPress there. After that, the usage should be more or less the basic WordPress stuff, which is outside the scope of this “tutorial”.

Just remember to update your WordPress plugins. I already have outdated plugins after a whopping five minutes of use.

In case you’re a financially responsible person, it is nice to know that DigitalOcean allows you to track quite easily how much money has gone into maintaining your blog. It actually reads right in the upper right corner of the Droplet management screen (the number is updated once a day).

Option C: The true tech approach
Step 1: Set up your own server HW
I’m not going to do this.

But that’s the process of doing simple things the hard way! Or one method of doing so. At least you save few bucks by hosting the server yourself (as opposed to buying premium WordPress elsewhere), and you already have a topic for your first blog post. Hopefully, the next one will have some actually informative content.

September 15, 2021 by ejaaskel Random ramblings 0

Get in Touch