Multi-platform development

The title of this article is perhaps a bit misleading: it’s not just about developing code which needs to run on different platforms, it’s also about using different programming languages.

Over the years, here at JeeLabs s/w development has always been about more than getting stuff working on one CPU: we want display and control tied to our laptops and mobile devices.

With µCs which collect and exchange data, multi-platform development is inevitable.

At some point, it might have been worth considering writing everything in C/C++, from the embedded µC to say a central Raspberry Pi, with server-side rendering of pages (as is still the model for PHP). But we’ve all seen the benefit of running something other than C on the server, and of course the browser world has been completely taken over by JavaScript. With rich “single-page applications” (SPA) taking over more and more of the user experience. And for good reason: we get astonishingly powerful and intuitive user interfaces (and eye-candy…).

It might seem that writing at least the server- and client-side using the same language is a good way forward. IOW: NodeJS on the server and JavaScript in the browser. And while JavaScript on a µC is definitely feasible, it’s not a convincing match in terms of performance and code size.

Meteor is one well-known example which tries to merge server- and client-side development, to the point that you can develop both sides as a single system, sharing code as much as possible.

There are some caveats with that approach: 1) developing in the same language on both sides ignores the fact that they deal with completely different problems (and that staying in the same context can actually be very confusing!), and 2) even NodeJS is a bit hefty for low-power Linux boxes such as Raspberry Pi or CHIP (it tends to fill up RAM when running for months on end).

So if JavaScript 1) can’t be avoided in the browser, 2) is not a great fit for low-power central always-on servers, and 3) doesn’t fit on low-end remote µC nodes, what’s the alternative?

My choices

Here’s the solution I’ve been moving towards for quite some time now:

1. Use Forth or C/C++ for remote embedded µCs
2. Use the Go language on the server side (it was actuallymade for that purpose)
3. Use JavaScript or a language which generates JavaScript in the browser

Both Forth and C/C++ fit easily, even on small µCs. They are both compiled code, so they don’t waste time running an interpreter. And they both go all the way down to the bare metal, so that full control of all the embedded hardware is always possible.

Go could be called “modern C/C++ for servers”: garbage collected, modern interface-based composition, and channels plus “goroutines” to tame multi-threading and asynchronous tasks. Without the race-condition risks that usually come with threads and interrupts. As bonus, it’s trivial toproduce stand-alone executables in Go for over a dozen platform / OS combinations.

And JavaScript… needs no explanation. Browsers dig JavaScript, period.

To borrow and re-purpose a phrase fromClojureScript somewhat:

• Forth and C/C++ scale downwards
• Go gets (parallel) work done
• JavaScript reaches

Which brings me to the main topic of this article: how to deal with all this, as a developer?

My setup

The answer is: with a bit of care, the entire development process can be streamlined to a very comfortable level. Here’s what it comes down to (a lot of this is personal preference, of course):

• Laptop - everything can be done with a single laptop (Win/Mac/Lin, your pick)
• Source code - all source code is tracked in git for total recall and tamper-proof storage
• Collaboration - use GitHub orGogs (self-hosted) for repositories and issue tracking
• Forth - source code in Git, µC interface using Folie and… look ma, no toolchain!
• C/C++ - the usual suspects: Arduino,STM32duino,libopencm3 - a plethora of choices
• Go - source code in Git, cross-compile to ARM with GOOS=linux GOARCH=arm go build
• JavaScript - probably via ClojureScript withReagent (i.e.React wrapper) andFigwheel

I very much prefer to work on Mac, although I’ve been using Linux (and Unix) longer than I can remember, and feel very much at home on its command line (GUIs are another story: too inconsistent and quirky for my tastes). Here’s the screen layout I use most often on my laptop:

For purely text-based work, I’m still considering using thei3 window manager, on a screen of its own in Linux. But as soon as browsers and other GUI tools are involved, the Mac has my vote.

There are two ways to run Linux on this same machine:

• Docker for Mac uses macOS’s newnative“Hypervisor” framework. It’s as close as you can get to running Linux code with macOS managing the machine and the GUI. There is no need to install a VM like VirtualBox, Parallels, or VMware - a fascinating trend!

• Running a Virtual Machine is the more traditional way to run macOS and Linux in parallel, with the ability to integrate a Linux GUI into the Mac’s. This leads to a more pure Linux setup.

In both cases, it’s easy to call into Linux using SSH and have it show up in a terminal window on the Mac side, as shown in the blue box above. Both can be configured to share areas of the underlying macOS file system, so you don’t have to copy stuff across - you can just “cd” to the right place. Source code can be edited on macOS and Linux will see the changes, and vice versa.

A virtual machine could also be used to run Windows (at the same time, if you have enough memory), but there’s little incentive for me to go there - other than to try out a Go application built for Windows, perhaps (using“GOOS=windows go build”).

Virtual machines have come a long way, at least with Parallels Desktop for Mac version 12. And it shows - on the USB side, for example: when plugging a USB device into the laptop, you can choose whether to connect it to macOS or to one of the running VMs. This now works so well that where FTDI caused infinite grief (in the form of kernel panics!) for many years, all of these issues go away when the device is tied to the Linux VM, making it do things the Mac (or rather: FTDI) never got right, like controlling both the DTR and the RTS pins over a serial USB link.

As a result, I can now launch Folie on Linux, while working on the Mac laptop, and all is well: Ctrl-C to reset the attached µC and “!u 6” to upload Mecrisp to a JeeNode Zero - \o/ - yeay!

Another aspect of Linux is that it can run qemu-user-static, which can launch an ARM-based executable on Intel hardware. This offers an astonishingly transparent form of emulation: once installed, Linux will automatically recognise the ARM file format and insert the Qemu layer.

Quite a convenient setup, all in all: macOS, Linux, Windows even, and the ability to run ARM executables, like the Linux build of Mecrisp Forth. All on one laptop, using a single file system.

Docker example

To give you an idea of what Docker can do, let’s create a “container” withqemu-user-static and Mecrisp for Linux (i.e. ARM), which will be launched every time we fire up the container:

$ cat Dockerfile
FROM ubuntu
RUN apt-get update
RUN apt-get install -y qemu-user-static
ADD mecrisp-stellaris-linux mecrisp
CMD "./mecrisp"
$

With that Dockerfile, and a copy of mecrisp-stellaris-linux in the current dir, we can do:

$ docker build .
Sending build context to Docker daemon 63.49 kB
Step 1/5 : FROM ubuntu
---> ebcd9d4fca80
Step 2/5 : RUN apt-get update
---> Running in 3c06495691ab
Get:1 http://archive.ubuntu.com/ubuntu xenial InRelease [247 kB]
[...etc...]
Step 3/5 : RUN apt-get install -y qemu-user-static
---> Running in e1651fff44b9
Reading package lists...
[...etc...]
The following NEW packages will be installed:
binfmt-support libpipeline1 qemu-user-static
0 upgraded, 3 newly installed, 0 to remove and 0 not upgraded.
Need to get 7879 kB of archives.
After this operation, 86.1 MB of additional disk space will be used.
[...etc...]
Step 4/5 : ADD mecrisp-stellaris-linux mecrisp
---> 88249491f198
Removing intermediate container 2148b8a50044
Step 5/5 : CMD "./mecrisp"
---> Running in 5e75eef61d34
---> 11e4ed2299dd
Removing intermediate container 5e75eef61d34
Successfully built 11e4ed2299dd
$

Lets give it a friendlier name:

$ docker tag 11e4ed2299dd mecrisp

The result is a “container” image, which is Docker’s terminology for a snapshot of the state after the above process completes (i.e. a download plus apt-get to install qemu-user-static). Now that we have such an image, we can launch the Mecrisp setup almost like any other application:

$ time docker run -it --rm mecrisp
Mecrisp-Stellaris RA 2.3.6 for Linux by Matthias Koch

real    0m1.203s
user    0m0.015s
sys     0m0.013s

That’s really all it takes: just over a second to bring up a Ubuntu context (within the Mac!), launch Mecrisp (emulating the ARM code via Qemu), stop the whole process again (since I typed ctrl-d right away), and clean everything up. Should also work on Docker for Windows.

So now there’s a way to try out Forth code from the Mac command line - look ma, no hands!