This week I had a chance to interview two of the developers of BUX, Luke Rohenaz and Siggi Oskarsson, on their newly released middleware suite of tools for the BSV blockchain (you can listen to the podcast here).

What is middleware for a blockchain, you ask? I’m glad you did. Middleware is traditionally the label given the tools that bridge the divide between apps and the operating system. In distributed applications, the middleware is often the layer that intermediates and facilitates communications between the distributed app and the backend database network or infrastructure. Therefore, in blockchain space, where the blockchain or the miners are the backend ‘database,’ then the middleware is the platform that abstracts the data storage layer so that applications can be written without having to deal with the details of the blockchain. Allowing them to focus more appropriately on their application business logic.

This is really a first for BSV, when compared to many of the other blockchains which have been blessed with copious amounts of funding, and is set to catapult BSV into the same league as Ethereum in terms of ease of use and available tools for developers to ‘dive in’ to create distributed blockchain applications.

The unique need for middleware on Bitcoin

Any developer working on blockchains is woefully aware that it isn’t just a database. Not only is the data unindexed and un-queryable, but getting the latest state of the data is always a chore and a manual process.

Additionally, most blockchains have their own language in order to program smart contracts. At the very least, middleware should be able to help with the indexing and state issues, allowing applications to create and query states from the blockchain much like how a traditional CRUD (Create, Read, Update, Delete) web application would.

With Bitcoin, the situation is made additionally challenging, given that any application written that leverages the blockchain inevitably will need to deal with user ‘wallets’ or key management. Indeed, this is one of the unique aspects of blockchain development. Each user must have their private keys managed such that the application can 1) maintain available balances of their tokens, and 2) be able to create and sign valid transactions on behalf of the users.

Until now, this has required that application developers on bitcoin must deal with UTXOs or unspent transaction outputs, otherwise known as ‘coins,’ and manage the keys that unlock them, paying to what is commonly referred to as ‘addresses.’ This is especially complex as the best practice for privacy on BSV is to avoid reusing addresses, so the use of deterministically generated addresses from master seed secrets (XPUBs), commonly known as HD-wallets, are employed.

Keeping track of which addresses and coins a user has access to has always been challenging and has necessitated relying on third-party indexing servers. This is the method most mobile wallets employ. For every mobile Bitcoin wallet, there is a centralized indexing service that is run by the wallet provider such that the mobile wallets can be kept up to date with the latest state of all the coins they have control over. While this works for simple wallets, if an application needed to do more customized types of spending or non-standard (read as ‘not a cash payment’) transactions, they would be forced to write their own customized indexing service.

This is where BUX stands in. It is a set of tools with a generic indexing service that developers can use and customize to index their own specific types of transactions without writing their own service and running their own fully verifying blockchain node. While the BUX agent does do validations, it only validates the transactions that the specific application it is interested in, not the whole universe of transactions.

What is interesting to an application may vary, depending on the specific users registered with it, a particular token type, or a particular locking script template. BUX achieves this universality by combining a RegEx pattern match and integration with HD-wallets, so all the application needs to do is to register the master public keys of its users, and BUX will do the rest, filtering, sorting, validating, preprocessing, and storing the txns that the application cares about. It also provides a ‘database view’ of the data so that the application can treat the BUX layer much like a traditional database, with rich querying capabilities.

Indexing is the missing link

If you think about it, an indexing server is really the missing link in blockchain services.

It is rarely spoken about explicitly as a separate component, but EVERY application on a blockchain needs one.

Everyone just ends up writing their own.

ElectrumSV, a popular desktop wallet, uses ElectrumX, an indexing server that is run by volunteers, to retrieve all relevant transactions for a user once they enter their wallet seed. As mentioned before, all mobile and web wallets run an indexing server in the backend for their users. Block Explorers, like WhatsOnChain.com and Blockchair.com, all run their own proprietary indexing services in the backend, which is how they can allow users to enter any txn id, or block id and be able to retrieve the information requested immediately. But retrieving the information per txn basis from a block explorer is not sufficient for wallets or apps because it is the total state derived from all the transactions concerned with a wallet that matters.

And this total state needs to be continuously updated when new transactions are seen on the blockchain.

While block explorers are good for low-level granular blockchain queries, only a holistic view that BUX as an UTXO-aware indexing and querying service will do for an app. With BUX, developers can simply ‘ask the database’ what the current state is for any given user wallet and be able to create transactions that operate on this state at the aggregate wallet level. BUX will handle all the details such as address rotation, change addresses, balance monitoring, etc.

Additionally, given the move away from raw addresses, which is P2PKH specific, since BUX uses a generic RegEx to recognize locking scripts, it has adopted the term “destination” instead of “address.”

Smart contracts, especially ones that do not involve an address, such as hash puzzles or shared secret locks, do not contain any signature checks, so paying to a “destination” is a more appropriate and generic term going forward.

Finally, given that BUX operates at the UTXO level, it has built-in support for tokens that use UTXOs directly, such as the STAS protocol. Interested developers who wish to create their own STAS token authentication service (or run a universal one for profit) can now simply run a BUX service customized to recognize their specific P2STAS template or token ID.

This greatly simplifies the work needed to deploy a STAS token project and showcases the flexibility of STAS.

This flexibility isn’t limited only to STAS, though. It is possible to use BUX as a scalable backend to write your own service for most of the other 2nd layer token protocols on BSV, though more work may be required given that layer 2 protocols may not use HD wallets the same way STAS and bitcoins do.

All said, if you are building a bitcoin application and are struggling to find out how to query for your related transactions, keep track of your balances, and don’t want to pay the ever-increasing costs to run a fully validating node that stores everything, or have to rely on a third party API which may not be able to serve your growing needs, BUX may just be the “missing link” you were looking for.

/Jerry Chan
WallStreetTechnologist

Don’t miss out the first ever BSV Global Blockchain Convention taking place at the Grand Hyatt in Dubai on May 24 – 26. Book your tickets today!

New to Bitcoin? Check out CoinGeek’s Bitcoin for Beginners section, the ultimate resource guide to learn more about Bitcoin—as originally envisioned by Satoshi Nakamoto—and blockchain.