Blockchain 2018 == NoSQL 2009

Disclaimer: The views in this article are my own, and do not necessarily represent the views of my employer.

Blockchain is to 2018 what NoSQL was to 2009.

• Large variety of implementations, each addressing a different part of the problem.
• Vastly different interfaces, despite overlapping terminology.
• Conferences where all of the major players are in attendance.
• Discussion of decades-old research finally applicable to modern software.
• A few key success stories.
• A few gross misapplications.

And, of course, haters from all corners, ready to tell you that you don’t need a blockchain, just like there were so many ready to tell you that you didn’t need NoSQL.

The haters are right, as they always are. You could have used literally any database to do the sorts of storage, and storage guarantees, that NoSQL gave you. You can sign data and write updates immutably to get the sorts of auditability and trust that a blockchain gives you.

This is why I find it interesting that many NoSQL leaders are such vocal Blockchain haters. They know what it’s like to say, “No, not everyone needs NoSQL, but it can provide value to some,” or, “No, they’re not new pieces, but their combination is greater than the sum of their parts.” Yet they’re fully willing to claim that Blockchain is less that worthless.

Maybe part of the problem is terminology, again. “What is NoSQL?” was an impossible question to answer. Beyond, “A database that doesn’t use SQL,” it meant something different to each person involved. “What is Blockchain?” is the same way. Is it the specific structure of “blocks of data, chained by cryptographic hashes”? Is it decentralized trust? If is “proof of $something” consensus? Is it cryptocurrency?

Both of these communities are personal to me. I was part of the NoSQL movement while working on Riak at Basho, and on CloudKit at Apple. I’m writing about Blockchain today, because for the last few months, I’ve been part of the Blockchain movement, working on a project at VMware . The NoSQL argument is over, so I won’t bother rehashing it. But, I would like to discuss here why I’ve found the Blockchain domain worth my time.

A large portion of my interest in Blockchain comes down to the specific project I’ve chosen to work on. My team at VMware understands that, just as NoSQL was a term used to describe tools for enabling scalable databases, “Blockchain” is a term used to describe tools for enabling distributed trust. The core tool often discussed, though it is really the combination of several technologies, is byzantine fault tolerant state machine replication.[1]

Yes, BFTSMR is “just” replication with consensus[2] and cryptography[3], and you could assemble a number of existing technologies to make your own. But, just like Dynamo was “just” a distributed hash table with logical clocks and gossip, the exact combination provides some rather different capabilities. Each of the components has seen major developments since the classic texts explored them:

• Consensus algorithms: with distributed systems becoming ever more common, we’ve had a renaissance of better understanding and implementation .
• Cryptographic signatures: hardware has many useful primitives built in to make this efficient, and things like threshold signatures have enabled new optimizations.
• State machine abstraction: it is easier than ever to embed an efficient interpreter in your system, to allow quick extensibility and experimentation.
• Networking: RDMA, public cloud, edge networking, and other new topologies beg for new communication patterns to harness their potential.
• Storage: flash is cheap, and any data model you want is readily available.

BFTSMR has been ignored for years (decades!) because it has been impractical for real systems. With the latest developments, even though it hasn’t reached the speed and efficiency of optimized central databases, it is starting to become a reasonable choice for some applications. If the abstractions commonly provided by Blockchain (like transactions recorded in an immutable log) allow people to take advantage of these classic good ideas and advancements, I am excited to help enable that.

There are some very real things to worry about in association with Blockchain. Cryptocurrencies are not investment strategies, and are not going to have any of the promised power-distribution and -equalization effects they peddle while their value and cost fluctuate so wildly. Control of existing networks is misunderstood. Proof of work is criminally wasteful. Current anonymization is so difficult that most attempts are trivially unmasked. But just as we didn’t let misunderstandings about consistency, early bugs with data loss, or questionable performance numbers bury the very real improvements brought by NoSQL, let’s not write off all of Blockchain just yet.

I’m very excited about the real business improvements that will be enabled by the technical innovations my team is making at VMware. If you would like to join us, please reach out either to one of us directly or through our careers portal .

[1] It’s scary how close you can come to an acronym: B yzantine fau L t t O lerant state ma CH ine replic A t I o N . Maybe that’s why Ethereum is a sta CK machine.

[2] Replication should imply consensus, but we saw how that went in NoSQL.

[3] Apologies for using “cryptography” to mean all things that may have cryptographic properties: hashes, signatures, obfuscation, …