Creating a Cryptocurrency in the Crystal Programming Language
source link: https://www.tuicool.com/articles/hit/RfeQ3aJ
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
This post is my attempt to understand the key aspects of the blockchain by exploring the internals. I started by reading the original bitcoin whitepaper , but I felt the only way to truly understand the blockchain is by building a new cryptocurrency from scratch.
That’s why I decided to create a cryptocurrency using the new Crystal programming language, and I dubbed it CrystalCoin . This article won’t discuss algorithm choices, hash difficulty, or similar topics. Instead, the focus will be on detailing a concrete example, which should provide a deeper, hands-on understanding of the strengths and limitations of blockchains.
If you haven’t read it yet, for more background on algos and hashing, I suggest you take a look at Demir Selmanovic’s article Cryptocurrency for Dummies: Bitcoin and Beyond .
Why I Chose the Crystal Programming Language
For a better demonstration, I wanted to use a productive language likeRuby without compromising the performance. Cryptocurrency has many time-consuming computations (namely mining and hashing ), and that’s why compiled languages such as C++ and Java are the languages of choice for building “real” cryptocurrencies. That being said, I wanted to use a language with a cleaner syntax so I could keep development fun and allow better readability. Crystal performance tends to be good anyway.
So, why did I decide to use the Crystal programming language ? Crystal’s syntax is heavily inspired by Ruby’s, so for me, it feels natural to read and easy to write. It has the added benefit of a lower learning curve, especially for experienced Ruby developers.
This is how the Crystal lang team puts it at their official website:
Fast as C, slick as Ruby.
However, unlike Ruby or JavaScript, which are interpreted languages, Crystal is a compiled language, making it much faster and offering a lower memory footprint. Under the hood, it uses LLVM for compiling to native code.
Crystal is also statically typed, which means the compiler will help you catch type errors in compile-time.
I’m not going to explain why I consider Crystal language awesome because it is beyond the scope of this article, but if you don’t find my optimism convincing, feel free to check out this article for a better overview of Crystal’s potential.
Note: This article assumes you already have a basic understanding of Object Oriented Programming (OOP).
Blockchain
So, what is a blockchain? It’s a list (chain) of blocks linked and secured by digital fingerprints (also known as crypto hashes).
The easiest way to think of it is as a linked list data structure. That being said, a linked list only required to have a reference to the previous element; a block must have an identifier depending on the previous block’s identifier, meaning that you cannot replace a block without recomputing every single block that comes after.
For now, think of blockchain as a series of blocks with some data linked with a chain, the chain being the hash of the previous block.
The entire blockchain would exist on each node that wants to interact with it, meaning it is copied on each one of the nodes in the network. No single server hosts it, which makes it decentralized .
Yes, this is weird compared to conventional centralized systems. Each of the nodes will have a copy of the entire blockchain (> 149 Gb in Bitcoin blockchain by December 2017 ).
Hashing and Digital Signature
So, what is this hash function? Think of the hash as a function, that returns a unique fingerprint when we give it a text/object. Even the smallest change in the input object would change the fingerprint dramatically.
There are different hashing algorithms, and in this article, we’ll be using the SHA256
hash algorithm, which is the one used in Bitcoin
.
Using SHA256
we’ll always result in 64 hexadecimal chars (256 bit) in length even if the input is less than 256 bits or much bigger than 256 bits:
Note with the last example, that just adding a .
(dot) resulted in a dramatic change in the hash.
Therefore, in a blockchain, the chain is built by passing the block data into a hashing algorithm that would generate a hash, which is linked to the next block, henceforth, forming a series of blocks linked with the hashes of the previous blocks.
Building a Cryptocurreny in Crystal
Now let’s start creating our Crystal project and build our SHA256
encryption.
Assuming you have Crystal programming language installed
, let’s create the skeleton of CrystalCoin
codebase by using Crystal’s built-in project tooling crystal init app [name]
:
% crystal init app crystal_coin create crystal_coin/.gitignore create crystal_coin/.editorconfig create crystal_coin/LICENSE create crystal_coin/README.md create crystal_coin/.travis.yml create crystal_coin/shard.yml create crystal_coin/src/crystal_coin.cr create crystal_coin/src/crystal_coin/version.cr create crystal_coin/spec/spec_helper.cr create crystal_coin/spec/crystal_coin_spec.cr Initialized empty Git repository in /Users/eki/code/crystal_coin/.git/
This command will create the basic structure for the project, with an already initialized git repository, license and readme files. It also comes with stubs for tests, and the shard.yml
file for describing the project and managing dependencies, also known as shards.
Let’s add the openssl
shard, which is needed to build SHA256
algorithm:
# shard.yml dependencies: openssl: github: datanoise/openssl.cr
Once that’s in, head back into your terminal and run crystal deps
. Doing this will pull down openssl
and its dependencies for us to utilize.
Now we have the required library installed in our code, let’s start by defining Block
class and then building the hash function.
# src/crystal_coin/block.cr require "openssl" module CrystalCoin class Block def initialize(data : String) @data = data end def hash hash = OpenSSL::Digest.new("SHA256") hash.update(@data) hash.hexdigest end end end puts CrystalCoin::Block.new("Hello, Cryptos!").hash
You can now test your application by running crystal run crystal src/crystal_coin/block.cr
from your terminal.
crystal_coin [master●] % crystal src/crystal_coin/block.cr 33eedea60b0662c66c289ceba71863a864cf84b00e10002ca1069bf58f9362d5
Designing our Blockchain
Each block is stored with a timestamp
and, optionally, an index
. In CrystalCoin
, we’re going to store both. To help ensure integrity throughout the blockchain, each block will have a self-identifying hash
. Like Bitcoin, each block’s hash will be a cryptographic hash of the block’s ( index
, timestamp
, data
, and the hash of the previous block’s hash previous_hash
). The data can be anything you want for now.
module CrystalCoin class Block property current_hash : String def initialize(index = 0, data = "data", previous_hash = "hash") @data = data @index = index @timestamp = Time.now @previous_hash = previous_hash @current_hash = hash_block end private def hash_block hash = OpenSSL::Digest.new("SHA256") hash.update("#{@index}#{@timestamp}#{@data}#{@previous_hash}") hash.hexdigest end end end puts CrystalCoin::Block.new(data: "Same Data").current_hash
In Crystal lang, we replace Ruby’s attr_accessor
, attr_getter
and attr_setter
methods with new keywords:
Another thing you may have noticed in Crystal is that we want to hint the compiler about specific types through our code. Crystal infers the types, but whenever you have ambiguity you can explicitly declare types as well. That’s why we added String
types for current_hash
.
Now let’s run block.cr
twice and note that the same data will generate different hashes because of the different timestamp
:
crystal_coin [master●] % crystal src/crystal_coin/block.cr 361d0df74e28d37b71f6c5f579ee182dd3d41f73f174dc88c9f2536172d3bb66 crystal_coin [master●] % crystal src/crystal_coin/block.cr b1fafd81ba13fc21598fb083d9429d1b8a7e9a7120dbdacc7e461791b96b9bf3
Now we have our block structure, but we’re creating a blockchain. We need to start adding blocks to form an actual chain. As I mentioned earlier, each block requires information from the previous block. But how does the first block in the blockchain get there? Well, the first block, or genesis
block, is a special block (a block with no predecessors). In many cases, it’s added manually or has unique logic allowing it to be added.
We’ll create a new function that returns a genesis block. This block is of index=0
, and it has an arbitrary data value and an arbitrary value in the previous_hash
parameter.
Let’s build or class method Block.first
that generates the genesis block:
module CrystalCoin class Block ... def self.first(data="Genesis Block") Block.new(data: data, previous_hash: "0") end ... end end
And let’s test it out using p CrystalCoin::Block.first
:
#<CrystalCoin::Block:0x10b33ac80 @current_hash="acb701a9b70cff5a0617d654e6b8a7155a8c712910d34df692db92455964d54e", @data="Genesis Block", @index=0, @timestamp=2018-05-13 17:54:02 +03:00, @previous_hash="0">
Now that we’re able to create a genesis block, we need a function that will generate succeeding blocks in the blockchain.
This function will take the previous block in the chain as a parameter, create the data for the block to be generated, and return the new block with the appropriate data. When new blocks hash information from previous blocks, the integrity of the blockchain increases with each new block.
An important consequence is that a block can’t be modified without changing the hash of every consecutive block. This is demonstrated in the example below. If the data in block 44 is changed from LOOP
to EAST
, all hashes of the consecutive blocks must be changed. This is because the hash of the block depends on the value of the previous_hash
(among other things).
If we didn’t do this, it would be easier for an outside party to change the data and replace our chain with an entirely new one of their own. This chain of hashes acts as cryptographic proof and helps ensure that once a block is added to the blockchain it cannot be replaced or removed. Let’s create the class method Block.next
:
module CrystalCoin class Block ... def self.next(previous_node, data = "Transaction Data") Block.new( data: "Transaction data number (#{previous_node.index + 1})", index: previous_node.index + 1, previous_hash: previous_hash.hash ) end ... end end
To try it out all together, we’ll create a simple blockchain. The first element of the list is the genesis block. And of course, we need to add the succeeding blocks. We’ll create five new blocks to demonstrate CrystalCoin
:
blockchain = [ CrystalCoin::Block.first ] previous_block = blockchain[0] 5.times do new_block = CrystalCoin::Block.next(previous_block: previous_block) blockchain << new_block previous_block = new_block end p blockchain
[#<CrystalCoin::Block:0x108c57c80 @current_hash= "df7f9d47bee95c9158e3043ddd17204e97ccd6e8958e4e41dacc7f0c6c0df610", @index=0, @previous_hash="0", @timestamp=2018-06-04 12:13:21 +03:00, @data="Genesis Block>, #<CrystalCoin::Block:0x109c89740 @current_hash= "d188fcddd056c044c783d558fd6904ceeb9b2366339af491a293d369de4a81f6", @index=1, @previous_hash= "df7f9d47bee95c9158e3043ddd17204e97ccd6e8958e4e41dacc7f0c6c0df610", @timestamp=2018-06-04 12:13:21 +03:00, @data="Transaction data number (1)">, #<CrystalCoin::Block:0x109cc8500 @current_hash= "0b71b61118b9300b4fe8cdf4a7cbcc1dac4da7a8a3150aa97630994805354107", @index=2, @previous_hash= "d188fcddd056c044c783d558fd6904ceeb9b2366339af491a293d369de4a81f6", @timestamp=2018-06-04 12:13:21 +03:00, @transactions="Transaction data number (2)">, #<CrystalCoin::Block:0x109ccbe40 @current_hash= "9111406deea4f07f807891405078a3f8537416b31ab03d78bda3f86d9ae4c584", @index=3, @previous_hash= "0b71b61118b9300b4fe8cdf4a7cbcc1dac4da7a8a3150aa97630994805354107", @timestamp=2018-06-04 12:13:21 +03:00, @transactions="Transaction data number (3)">, #<CrystalCoin::Block:0x109cd0980 @current_hash= "0063bfc5695c0d49b291a8813c566b047323f1808a428e0eb1fca5c399547875", @index=4, @previous_hash= "9111406deea4f07f807891405078a3f8537416b31ab03d78bda3f86d9ae4c584", @timestamp=2018-06-04 12:13:21 +03:00, @transactions="Transaction data number (4)">, #<CrystalCoin::Block:0x109cd0100 @current_hash= "00a0c70e5412edd7389a0360b48c407ce4ddc8f14a0bcf16df277daf3c1a00c7", @index=5, @previous_hash= "0063bfc5695c0d49b291a8813c566b047323f1808a428e0eb1fca5c399547875", @timestamp=2018-06-04 12:13:21 +03:00, @transactions="Transaction data number (5)">
Proof-of-Work
A Proof of Work algorithm (PoW) is how new blocks are created or mined on the blockchain. The goal of PoW is to discover a number which solves a problem. The number must be difficult to find but easy to verify computationally by anyone on the network. This is the core idea behind Proof of Work.
Let’s demonstrate with an example to make sure everything is clear. We’ll assume that the hash of some integer x multiplied by another y must start with 00
. So:
hash(x * y) = 00ac23dc...
And for this simplified example, let’s fix x=5
and implement this in Crystal:
x = 5 y = 0 while hash((x*y).to_s)[0..1] != "00" y += 1 end puts "The solution is y = #{y}" puts "Hash(#{x}*#{y}) = #{hash((x*y).to_s)}"
Let’s run the code:
crystal_coin [master●●] % time crystal src/crystal_coin/pow.cr The solution is y = 530 Hash(5*530) = 00150bc11aeeaa3cdbdc1e27085b0f6c584c27e05f255e303898dcd12426f110 crystal src/crystal_coin/pow.cr 1.53s user 0.23s system 160% cpu 1.092 total
As you can see this number y=530
was hard to find (brute-force) but easy to verify using the hash function.
Why bother with this PoW algorithm? We don’t we just create one hash per block and that’s it. A hash must be valid
. In our case, a hash will be valid if the first two characters of our hash are 00
. If our hash starts with 00......
, it is considered valid. This is called the difficulty
. The higher the difficulty, the longer it takes to get a valid hash.
But, if the hash is not valid the first time, something must change in the data we use. If we use the same data over and over, we will get the same hash over and over and our hash will never be valid. We use something called nonce
in our hash (in our previous example it’s the y
). It is simply a number that we increment each time the hash is not valid. We get our data (date, message, previous hash, index) and a nonce of 1. If the hash we get with these is not valid, we try with a nonce of 2. And we increment the nonce until we get a valid hash.
In Bitcoin, the Proof of Work algorithm is called Hashcash . Let’s add a proof-of-work to our Block class and start mining to find the nonce. We’ll use a hard-coded difficulty of two leading zeros ‘00’:
Now let’s redesign our Block class to support that. Our CrystalCoin
Block will contain the following attributes:
1) index: indicates the index of the block ex: 0,1 2) timestamp: timestamp in epoch, number of seconds since 1 Jan 1970 3) data: the actual data that needs to be stored on the blockchain. 4) previous_hash: the hash of the previous block, this is the chain/link between the blocks 5) nonce: this is the number that is to be mined/found. 6) current_hash: The hash value of the current block, this is generated by combining all the above attributes and passing it to a hashing algorithm
I’ll create a separate module to do the hashing and find the nonce
so we keep our code clean and modular. I’ll call it proof_of_work.cr
:
require "openssl" module CrystalCoin module ProofOfWork private def proof_of_work(difficulty = "00") nonce = 0 loop do hash = calc_hash_with_nonce(nonce) if hash[0..1] == difficulty return nonce else nonce += 1 end end end private def calc_hash_with_nonce(nonce = 0) sha = OpenSSL::Digest.new("SHA256") sha.update("#{nonce}#{@index}#{@timestamp}#{@data}#{@previous_hash}") sha.hexdigest end end end
Our Block
class would look something like:
require "./proof_of_work" module CrystalCoin class Block include ProofOfWork property current_hash : String property index : Int32 property nonce : Int32 property previous_hash : String def initialize(index = 0, data = "data", previous_hash = "hash") @data = data @index = index @timestamp = Time.now @previous_hash = previous_hash @nonce = proof_of_work @current_hash = calc_hash_with_nonce(@nonce) end def self.first(data = "Genesis Block") Block.new(data: data, previous_hash: "0") end def self.next(previous_block, data = "Transaction Data") Block.new( data: "Transaction data number (#{previous_block.index + 1})", index: previous_block.index + 1, previous_hash: previous_block.current_hash ) end end end
Few things to note about Crystal code and Crystal language examples in general. In Crystal, methods are public by default. Crystal requires each private method to be prefixed with the private keyword which could be confusing coming from Ruby.
You may have noticed that Crystal’s Integer types there are Int8
, Int16
, Int32
, Int64
, UInt8
, UInt16
, UInt32
, or UInt64
compared to Ruby’s Fixnum
. true
and false
are values in the Bool
class rather than values in classes TrueClass
or FalseClass
in Ruby.
Crystal has optional and named method arguments as core language features, and does not require writing special code for handling the arguments which is pretty cool. Check out Block#initialize(index = 0, data = "data", previous_hash = "hash")
and then calling it with something like Block.new(data: data, previous_hash: "0")
.
For a more detailed list of differences between Crystal and Ruby programming language check out Crystal for Rubyists .
Now, let’s try to create five transactions using:
blockchain = [ CrystalCoin::Block.first ] puts blockchain.inspect previous_block = blockchain[0] 5.times do |i| new_block = CrystalCoin::Block.next(previous_block: previous_block) blockchain << new_block previous_block = new_block puts new_block.inspect end
[#<CrystalCoin::Block:0x108f8fea0 @current_hash="0088ca080a49334e1cb037ed4c42795d635515ef1742e6bcf439bf0f95711759", @index=0, @nonce=17, @timestamp=2018-05-14 17:20:46 +03:00, @data="Genesis Block", @previous_hash="0">] #<CrystalCoin::Block:0x108f8f660 @current_hash="001bc2b04d7ad8ef25ada30e2bde19d7bbbbb3ad42348017036b0d4974d0ccb0", @index=1, @nonce=24, @timestamp=2018-05-14 17:20:46 +03:00, @data="Transaction data number (1)", @previous_hash="0088ca080a49334e1cb037ed4c42795d635515ef1742e6bcf439bf0f95711759"> #<CrystalCoin::Block:0x109fc5ba0 @current_hash="0019256c998028111838b872a437cd8adced53f5e0f8f43388a1dc4654844fe5", @index=2, @nonce=61, @timestamp=2018-05-14 17:20:46 +03:00, @data="Transaction data number (2)", @previous_hash="001bc2b04d7ad8ef25ada30e2bde19d7bbbbb3ad42348017036b0d4974d0ccb0"> #<CrystalCoin::Block:0x109fdc300 @current_hash="0080a30d0da33426a1d4f36d870d9eb709eaefb0fca62cc68e497169c5368b97", @index=3, @nonce=149, @timestamp=2018-05-14 17:20:46 +03:00, @data="Transaction data number (3)", @previous_hash="0019256c998028111838b872a437cd8adced53f5e0f8f43388a1dc4654844fe5"> #<CrystalCoin::Block:0x109ff58a0 @current_hash="00074399d51c700940e556673580a366a37dec16671430141f6013f04283a484", @index=4, @nonce=570, @timestamp=2018-05-14 17:20:46 +03:00, @data="Transaction data number (4)", @previous_hash="0080a30d0da33426a1d4f36d870d9eb709eaefb0fca62cc68e497169c5368b97"> #<CrystalCoin::Block:0x109fde120 @current_hash="00720bb6e562a25c19ecd2b277925057626edab8981ff08eb13773f9bb1cb842", @index=5, @nonce=475, @timestamp=2018-05-14 17:20:46 +03:00, @data="Transaction data number (5)", @previous_hash="00074399d51c700940e556673580a366a37dec16671430141f6013f04283a484">
See the difference? Now all hashes start with 00
. That’s the magic of proof-of-work. Using ProofOfWork
we found the nonce
and proof is the hash with the matching difficulty, that is, the two leading zeros 00
.
Note with the first block we created, we tried 17 nonces until finding the matching lucky number:
Block Loops / Number of Hash Calculations #0 17 #1 24 #2 61 #3 149 #4 570 #5 475
Now let’s try a difficulty of four leading zeros ( difficulty="0000"
):
In the first block tried 26762 nonces (compare 17 nonces with difficulty ‘00’) until finding the matching lucky number.
Our Blockchain as an API
So far, so good. We created our simple blockchain and it was relatively easy to do. But the problem here is that CrystalCoin
can only run on one single machine (it’s not distributed/decentralized).
From now on, we’ll start using JSON data for CrystalCoin
. The data will be transactions, so each block’s data field will be a list of transactions.
Each transaction will be a JSON object detailing the sender
of the coin, the receiver
of the coin, and the amount
of CrystalCoin that is being transferred:
{ "from": "71238uqirbfh894-random-public-key-a-alkjdflakjfewn204ij", "to": "93j4ivnqiopvh43-random-public-key-b-qjrgvnoeirbnferinfo", "amount": 3 }
A few modifications to our Block
class to support the new transaction
format (previously called data
). So, just to avoid confusion and maintain consistency, we’ll be using the term transaction
to refer to data
posted in our example application from now on.
We’ll introduce a new simple class Transaction
:
module CrystalCoin class Block class Transaction property from : String property to : String property amount : Int32 def initialize(@from, @to, @amount) end end end end
The transactions are packed into blocks, so a block can contain just one or numerous transactions. Blocks containing the transactions are generated frequently and added to the blockchain.
The blockchain is supposed to be a collection of blocks. We can store all of the blocks in the Crystal list, and that’s why we introduce the new class Blockchain
:
Blockchain
will have chain
and uncommitted_transactions
arrays. The chain
will include all the mined blocks in the blockchain, and uncommitted_transactions
will have all the transactions that have not been added to the blockchain (still not mined). Once we initialize Blockchain
, we create the genesis block using Block.first
and add it to chain
array, and we add an empty uncommitted_transactions
array.
We will create Blockchain#add_transaction
method to add transactions to uncommitted_transactions
array.
Let’s build our new Blockchain
class:
require "./block" require "./transaction" module CrystalCoin class Blockchain getter chain getter uncommitted_transactions def initialize @chain = [ Block.first ] @uncommitted_transactions = [] of Block::Transaction end def add_transaction(transaction) @uncommitted_transactions << transaction end end end
In Block
class we will start using transactions
instead of data
:
module CrystalCoin class Block include ProofOfWork def initialize(index = 0, transactions = [] of Transaction, previous_hash = "hash") @transactions = transactions ... end .... def self.next(previous_block, transactions = [] of Transaction) Block.new( transactions: transactions, index: previous_block.index + 1, previous_hash: previous_block.current_hash ) end end end
Now that we know what our transactions will look like, we need a way of adding them to one of the computers in our blockchain network, called a node
. To do that, we’ll create a simple HTTP server.
We’ll create four endpoints:
-
[POST]
/transactions/new
: to create a new transaction to a block -
[GET]
/mine
: to tell our server to mine a new block. -
[GET]
/chain
: to return the full blockchain inJSON
format. -
[GET]
/pending
: to return the pending transactions (uncommitted_transactions
).
We’re going to use Kemal web framework. It’s a micro-framework that makes it easy to map endpoints to Crystal functions. Kemal is heavily influenced by Sinatra for Rubyists and works in a very similar way. If you are looking for Ruby on Rails equivalent then check out Amber .
Our server will form a single node in our blockchain network. Let’s first add Kemal
to the shard.yml
file as a and install the dependency:
dependencies: kemal: github: kemalcr/kemal
Now let’s build the skeleton of our HTTP server:
# src/server.cr require "kemal" require "./crystal_coin" # Generate a globally unique address for this node node_identifier = UUID.random.to_s # Create our Blockchain blockchain = Blockchain.new get "/chain" do "Send the blockchain as json objects" end get "/mine" do "We'll mine a new Block" end get "/pending" do "Send pending transactions as json objects" end post "/transactions/new" do "We'll add a new transaction" end Kemal.run
And run the server:
crystal_coin [master●●] % crystal run src/server.cr [development] Kemal is ready to lead at http://0.0.0.0:3000
Let’s make sure the server is working fine:
% curl http://0.0.0.0:3000/chain Send the blockchain as json objects%
So far so good. Now, we can proceed with implementing each of the endpoints. Let’s start by implementing /transactions/new
and pending
end-points:
get "/pending" do { transactions: blockchain.uncommitted_transactions }.to_json end post "/transactions/new" do |env| transaction = CrystalCoin::Block::Transaction.new( from: env.params.json["from"].as(String), to: env.params.json["to"].as(String), amount: env.params.json["amount"].as(Int64) ) blockchain.add_transaction(transaction) "Transaction #{transaction} has been added to the node" end
Straightforward implementation. We just create a CrystalCoin::Block::Transaction
object and add the transaction to the uncommitted_transactions
array using Blockchain#add_transaction
.
At the moment, the transactions are initially stored in a pool of uncommitted_transactions
. The process of putting the unconfirmed transactions in a block and computing Proof of Work (PoW) is known as the mining
of blocks. Once the nonce
satisfying our constraints is figured out, we can say that a block has been mined, and the new block is put into the blockchain.
In CrystalCoin
, we’ll use the simple Proof-of-Work algorithm we created earlier. To create a new block, a miner’s computer will have to:
chain uncommitted_transactions Block.next chain uncommitted_transactions
So to implement /mine
end-point, let’s first implement the above steps in Blockchain#mine
:
module CrystalCoin class Blockchain include Consensus BLOCK_SIZE = 25 ... def mine raise "No transactions to be mined" if @uncommitted_transactions.empty? new_block = Block.next( previous_block: @chain.last, transactions: @uncommitted_transactions.shift(BLOCK_SIZE) ) @chain << new_block end end end
We make sure first we have some pending transactions to mine. Then we get the last block using @chain.last
, and the first 25
transactions to be mined (we are using Array#shift(BLOCK_SIZE)
to return an array of the first 25 uncommitted_transactions
, and then remove the elements starting at index 0).
Now let’s implement /mine
end-point:
get "/mine" do blockchain.mine "Block with index=#{blockchain.chain.last.index} is mined." end
And for /chain
end-point:
get "/chain" do { chain: blockchain.chain }.to_json end
Interacting With our Blockchain
We’ll be using cURL
to interact with our API over a network.
First, let’s fire up the server:
crystal_coin [master] % crystal run src/server.cr [development] Kemal is ready to lead at http://0.0.0.0:3000
Then let’s create two new transactions by making a POST
request to http://localhost:3000/transactions/new
with a body containing our transaction structure:
crystal_coin [master●] % curl -X POST http://0.0.0.0:3000/transactions/new -H "Content-Type: application/json" -d '{"from": "eki", "to":"iron_man", "amount": 1000}' Transaction #<CrystalCoin::Block::Transaction:0x10c4159f0 @from="eki", @to="iron_man", @amount=1000_i64> has been added to the node% crystal_coin [master●] % curl -X POST http://0.0.0.0:3000/transactions/new -H "Content-Type: application/json" -d '{"from": "eki", "to":"hulk", "amount": 700}' Transaction #<CrystalCoin::Block::Transaction:0x10c415810 @from="eki", @to="hulk", @amount=700_i64> has been added to the node%
Now let’s list the pending transactions (i.e. transactions that have not been added to the block yet):
crystal_coin [master●] % curl http://0.0.0.0:3000/pendings { "transactions":[ { "from":"ekis", "to":"huslks", "amount":7090 }, { "from":"ekis", "to":"huslks", "amount":70900 } ] }
As we can see, the two transactions we created earlier have been added to uncommitted_transactions
.
Now let’s mine
the two transactions by making a GET
request to http://0.0.0.0:3000/mine
:
crystal_coin [master●] % curl http://0.0.0.0:3000/mine Block with index=1 is mined.
Looks like we successfully mined the first block and added it to our chain
. Let’s double check our chain
and if it includes the mined block:
crystal_coin [master●] % curl http://0.0.0.0:3000/chain { "chain": [ { "index": 0, "current_hash": "00d469d383005b4303cfa7321c02478ce76182564af5d16e1a10d87e31e2cb30", "nonce": 363, "previous_hash": "0", "transactions": [ ], "timestamp": "2018-05-23T01:59:52+0300" }, { "index": 1, "current_hash": "003c05da32d3672670ba1e25ecb067b5bc407e1d5f8733b5e33d1039de1a9bf1", "nonce": 320, "previous_hash": "00d469d383005b4303cfa7321c02478ce76182564af5d16e1a10d87e31e2cb30", "transactions": [ { "from": "ekis", "to": "huslks", "amount": 7090 }, { "from": "ekis", "to": "huslks", "amount": 70900 } ], "timestamp": "2018-05-23T02:02:38+0300" } ] }
Consensus and Decentralization
This is cool. We got ourselves a basic blockchain that accepts transactions and allows us to mine new blocks. But the code that we’ve implemented till now is meant to run on a single computer, while the whole point of blockchains is that they should be decentralized. But if they’re decentralized, how we ensure that they all reflect the same chain?
This is the problem of Consensus
.
We’ll have to implement a consensus algorithm if we want more than one node in our network.
Registering New Nodes
To implement a consensus algorithm, we need a way to let a node know about neighboring nodes on the network. Each node on our network should keep a registry of other nodes on the network. Therefore, we’ll need more endpoints:
/nodes/register /nodes/resolve
We need to modify our blockchain’s constructor and provide a method for registering nodes:
--- a/src/crystal_coin/blockchain.cr +++ b/src/crystal_coin/blockchain.cr @@ -7,10 +7,12 @@ module CrystalCoin getter chain getter uncommitted_transactions + getter nodes def initialize @chain = [ Block.first ] @uncommitted_transactions = [] of Block::Transaction + @nodes = Set(String).new [] of String end def add_transaction(transaction)
Note that we’ve used a Set
data structure with String
type to hold the list of nodes. This is a cheap way of ensuring that the addition of new nodes is idempotent and that no matter how many times we add a specific node, it appears exactly once.
Now let’s add a new module to Consensus
and implement the first method register_node(address)
:
require "uri" module CrystalCoin module Consensus def register_node(address : String) uri = URI.parse(address) node_address = "#{uri.scheme}:://#{uri.host}" node_address = "#{node_address}:#{uri.port}" unless uri.port.nil? @nodes.add(node_address) rescue raise "Invalid URL" end end end
The register_node
function, will parse the URL of the node and format it.
And here let’s create /nodes/register
end-point:
post "/nodes/register" do |env| nodes = env.params.json["nodes"].as(Array) raise "Empty array" if nodes.empty? nodes.each do |node| blockchain.register_node(node.to_s) end "New nodes have been added: #{blockchain.nodes}" end
Now with this implementation, we might face a problem with multiple nodes. The copy of chains of a few nodes can differ. In that case, we need to agree upon some version of the chain to maintain the integrity of the entire system. We need to achieve consensus.
To resolve this, we’ll make the rule that the longest valid chain is the one to be used. Using this algorithm, we reach consensus amongst the nodes in our network. The reason behind this approach is that the longest chain is a good estimate of the most amount of work done.
module CrystalCoin module Consensus ... def resolve updated = false @nodes.each do |node| node_chain = parse_chain(node) return unless node_chain.size > @chain.size return unless valid_chain?(node_chain) @chain = node_chain updated = true rescue IO::Timeout puts "Timeout!" end updated end ... end end
Bear in mind that resolve
is a method which loops through all our neighboring nodes, downloads their chains and verifies them using the valid_chain?
method. If a valid chain is found, whose length is greater than ours, we replace ours.
Now let’s implement parse_chain()
and valid_chain?()
private methods:
module CrystalCoin module Consensus ... private def parse_chain(node : String) node_url = URI.parse("#{node}/chain") node_chain = HTTP::Client.get(node_url) node_chain = JSON.parse(node_chain.body)["chain"].to_json Array(CrystalCoin::Block).from_json(node_chain) end private def valid_chain?(node_chain) previous_hash = "0" node_chain.each do |block| current_block_hash = block.current_hash block.recalculate_hash return false if current_block_hash != block.current_hash return false if previous_hash != block.previous_hash return false if current_block_hash[0..1] != "00" previous_hash = block.current_hash end return true end end end
For parse_chain()
we:
-
Issue a
GET
HTTP request usingHTTP::Client.get
to/chain
end-point. -
Parse the
/chain
JSON response usingJSON.parse
. -
Extract an array of
CrystalCoin::Block
objects from the JSON blob that was returned usingArray(CrystalCoin::Block).from_json(node_chain)
.
There is more than one way of parsing JSON in Crystal. The preferred method is to use Crystal’s super-handy JSON.mapping(key_name: Type)
functionality that gives us the following:
Class.from_json instance.to_json
In our case, we had to define JSON.mapping
in CrystalCoin::Block
object, and we removed property
usage in the class, like so:
module CrystalCoin class Block JSON.mapping( index: Int32, current_hash: String, nonce: Int32, previous_hash: String, transactions: Array(Transaction), timestamp: Time ) ... end end
Now for Blockchain#valid_chain?
, we iterate through all of the blocks, and for each we:
-
Recalculate the hash for the block using
Block#recalculate_hash
and check that the hash of the block is correct:
module CrystalCoin class Block ... def recalculate_hash @nonce = proof_of_work @current_hash = calc_hash_with_nonce(@nonce) end end end
- Check each of the blocks linked correctly with their previous hashes.
-
Check the block’s hash is valid for the number of zeros (
difficulty
in our case00
).
And finally we implement /nodes/resolve
end-point:
get "/nodes/resolve" do if blockchain.resolve "Successfully updated the chain" else "Current chain is up-to-date" end end
It’s done! You can find the final code on GitHub.
The structure of our project should look like this:
crystal_coin [master●] % tree src/ src/ ├── crystal_coin │ ├── block.cr │ ├── blockchain.cr │ ├── consensus.cr │ ├── proof_of_work.cr │ ├── transaction.cr │ └── version.cr ├── crystal_coin.cr └── server.cr
Let’s Try it Out
-
Grab a different machine, and run different nodes on your network. Or spin up processes using different ports on the same machine. In my case, I created two nodes on my machine, on a different port to have two nodes:
http://localhost:3000
andhttp://localhost:3001
. - Register the second node address to the first node using:
crystal_coin [master●●] % curl -X POST http://0.0.0.0:3000/nodes/register -H "Content-Type: application/json" -d '{"nodes": ["http://0.0.0.0:3001"]}' New nodes have been added: Set{"http://0.0.0.0:3001"}%
- Let’s add a transaction to the second node:
crystal_coin [master●●] % curl -X POST http://0.0.0.0:3001/transactions/new -H "Content-Type: application/json" -d '{"from": "eqbal", "to":"spiderman", "amount": 100}' Transaction #<CrystalCoin::Block::Transaction:0x1039c29c0> has been added to the node%
- Let’s mine transactions into a block at the second node:
crystal_coin [master●●] % curl http://0.0.0.0:3001/mine Block with index=1 is mined.%
- At this point, the first node has only one block (genesis block), and the second node has two nodes (genesis and the mined block):
crystal_coin [master●●] % curl http://0.0.0.0:3000/chain {"chain":[{"index":0,"current_hash":"00fe9b1014901e3a00f6d8adc6e9d9c1df03344dda84adaeddc8a1c2287fb062","nonce":157,"previous_hash":"0","transactions":[],"timestamp":"2018-05-24T00:21:45+0300"}]}%
crystal_coin [master●●] % curl http://0.0.0.0:3001/chain {"chain":[{"index":0,"current_hash":"007635d82950bc4b994a91f8b0b20afb73a3939e660097c9ea8416ad614faf8e","nonce":147,"previous_hash":"0","transactions":[],"timestamp":"2018-05-24T00:21:38+0300"},{"index":1,"current_hash":"00441a4d9a4dfbab0b07acd4c7639e53686944953fa3a6c64d2333a008627f7d","nonce":92,"previous_hash":"007635d82950bc4b994a91f8b0b20afb73a3939e660097c9ea8416ad614faf8e","transactions":[{"from":"eqbal","to":"spiderman","amount":100}],"timestamp":"2018-05-24T00:23:57+0300"}]}%
- Our goal is to update the chain in the first node to include the newly generated block at the second one. So let’s resolve the first node:
crystal_coin [master●●] % curl http://0.0.0.0:3000/nodes/resolve Successfully updated the chain%
Let’s check if the chain in the first node has updated:
crystal_coin [master●●] % curl http://0.0.0.0:3000/chain {"chain":[{"index":0,"current_hash":"007635d82950bc4b994a91f8b0b20afb73a3939e660097c9ea8416ad614faf8e","nonce":147,"previous_hash":"0","transactions":[],"timestamp":"2018-05-24T00:21:38+0300"},{"index":1,"current_hash":"00441a4d9a4dfbab0b07acd4c7639e53686944953fa3a6c64d2333a008627f7d","nonce":92,"previous_hash":"007635d82950bc4b994a91f8b0b20afb73a3939e660097c9ea8416ad614faf8e","transactions":[{"from":"eqbal","to":"spiderman","amount":100}],"timestamp":"2018-05-24T00:23:57+0300"}]}%
Hooray! Our Crystal language example works like a charm, and I hope you found this lengthy tutorial crystal clear, pardon the pun.
Wrapping Up
This Crystal language tutorial covered the fundamentals of a public blockchain. If you followed along, you implemented a blockchain from scratch and built a simple application allowing users to share information on the blockchain.
We’ve made a fairly sized blockchain at this point. Now, CrystalCoin
can be launched on multiple machines to create a network, and real CrystalCoins
can be mined.
I hope that this has inspired you to create something new, or at the very least take a closer look at Crystal programming.
Note: The code in this tutorial is not ready for real-world use. Please refer to this as a general guide only.
Understanding the Basics
What are the uses for Crystal programming language?
Crystal is a generic language. You can do whatever you can do in Ruby using Crystal, with better performance and less memory use. One selling point for Crystal is the ease with which you can interface with C libraries, as Crystal allows you to bind to existing C libraries without writing a single line in C.
Why should I learn Crystal programming language?
Crystal is a programming language that can be easily understood by humans and compiles to fast programs. It's a statically-typed, compiled language, that achieves performance close to c/c++ while having a syntax as readable as Ruby.
What’s the difference between Ruby and Crystal?
Crystal has a Ruby-like syntax, but it’s a different language, not a Ruby implementation. Because it's a compiled, statically typed language, the language has some differences when compared to Ruby. Coming from Ruby, Crystal has a low learning curve.
What about frameworks?
If you love the completeness of Rails, you'll feel right at home with the Amber framework. If the simplicity and easy customization of Sinatra more your thing, you'll find that simplicity with Kemal.
Is Crystal a promising programming language?
It is one of the more promising programming languages today. It appears to have shattered the barrier between syntactically sweet interpreted/dynamic languages like Ruby/Python, which are loved for their readability and ease-of-use, and the raw horsepower of C/C++ and low-level systems languages.
Recommend
About Joyk
Aggregate valuable and interesting links.
Joyk means Joy of geeK