Create Once, Use Twice — How To Stream Your Espruino Project’s User Interface to the Web

Espruino is my go-to platform for developing IoT projects. It is a small, embedded JavaScript interpreter that can run on variety of devices. I used it in my In-Real-Life Chrome T-Rex Game, as well as my Smart Air Pressure Sensor project, and keep using it almost for every new project I make.

A few months ago, I discovered the Pixl.js — an Espruino board that combines Bluetooth Low Energy connectivity together with a low-power LCD display, and has an Arduino-compatible form factor. I use it in my DiY Rock Tumbler project, and also for the Trumpet Playing Robot I’m currently trying to build. I use it so much I even managed to fry the board (and then bring it back from the dead).

A User Interface for Your Projects

In addition to the LCD screen, Pixl.js also features 4 push buttons, placed next to the edges of the screen, which make it ideal for designing simple control interfaces, display various statistics and providing menus to configure the hardware project you are building.

For instance, in the rock tumbler, the screen would display the current speed and the total number of revolutions so far, and the buttons would allow to make it go faster / slower, switch the direction, and to start / stop the motors:

The User Interface for Controlling the Rock Tumbler Project

On top of that, Espruino also provides a simple Menu Library that enables creation of hierarchical menus:

Espruino Menus (and a T-Rex)

As you can imagine, this is very convenient — especially when you consider the small amount of code needed for creating this menu:

Remote Controlling The Device

I really like how simple it is to create a small control panel with a graphical user interface for your project. However, once you want to add some form of remote control through Bluetooth, you find yourself in a situation where you suddenly need to write a considerable amount of code. You need to write some Espruino code that exposes specific Bluetooth services, and then also write an application (Web or Native) that will connect to your device and control it remotely.

In other words, if you want to control your project both through the physical buttons and LCD display and remotely over Bluetooth — you have to work twice, write a bunch of glue code, and design two different user interfaces.

Thus, I started experimenting with ways to stream the content of the LCD display to a web page running on a computer / mobile device, over Web Bluetooth. By streaming the content of the display, you can reuse the existing user interface without writing any additional code. I call this experimental project espruino-mirror.

How To Use Espruino-Mirror?

Just copy the mirroring code into your project’s code, and upload it to Espruino. Then open the Online Espruino Viewer, click the “Connect” button and connect to your device. As soon as Espruino updates the screen, you should also see the new screen content updated in the web page:

Espruino Mirror Showing displaying the Morphing Clock app

Espruino-mirror exposes a Bluetooth service (whose number is 0xfeed), with three characteristics: 0xfe01 and 0xfe02 contain the width and the height of the display, respectively, and then 0xfeed is used to stream bits of the display whenever the code refreshes the screen.

If you need a quick introduction for Bluetooth Low Energy services and Characteristics, check out the “Bluetooth Low Energy 101” section in my Reverse Engineering a Bluetooth Light Bulb post.

Streaming the display works by hooking to the g.flip() method of the Graphics library. This method is called whenever the code wants to update the screen with the current contents of the display buffer. We simply replace this method with our own implementation:

The code splits the screen buffer into chunks and sending each chunk over Bluetooth characteristic 0xfeed. The first chunk is 19 bytes long (this is how indicate to the receiver that this chunk goes at the top of the string), which each subsequent chunk being 20 bytes long.

The reason for splitting the buffer into 20 byte chunks is that this is the largest amount of data you can transmit in one Bluetooth packet by default. The fastest connection interval for Bluetooth Low Energy is 7.5 milliseconds, so we can transfer at most one packet every 7.5 milliseconds, or about 133 packets per seconds. Therefore, our bandwidth maxes at around 133*20, or 2660 bytes per second. Luckily enough, the Pixl.js screen buffer is only 1024 bytes, so we can achieve an acceptable update rate.

The latency is noticeable— the physical display is only updated after the current frame has been sent

Performance and Power Consumption

Sending the entire screen buffer down the Bluetooth Low Energy link does consume its share of CPU power, and also most of the available BLE bandwidth. I included a simple optimization which only starts streaming when someone is connected.

Bluetooth 4.2 introduced support for larger packets, allowing up to 244 bytes of user data in a single packet. Using these larger packets, we can achieve about 12 times faster refresh rate. However, by looking at Espruino’s source code, it seems like the larger packet sizes are not supported yet. At least we know the hardware is capable of that!

Tinkering with the Menu Example

Evolving Espruino-Mirror

I had a bunch of ideas how to evolve Espruino-Mirror. The first thing I’d probably do would be adding 4 buttons to the Web interface, so it could also be used to provide input to the device, and not only view its screen. Another improvement would be to publish the code as an Espruino module, making it easier to consume it in your projects.

Hopefully, at some point, Espruino adds support for increasing the packet size, so we can also get decent screen refresh rates.

I’d love to see other people using Espruino-mirror in their projects and sharing the experiences. You can leave a comment below, or open an issue in the GitHub Repository of the project. Pull Requests are welcome!