5

Let’s Create Our Own Cryptocurrency | cranklin.com

 4 years ago
source link: https://cranklin.wordpress.com/2017/07/11/lets-create-our-own-cryptocurrency/
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
neoserver,ios ssh client
Let’s Create Our Own Cryptocurrency

I’ve been itching to build my own cryptocurrency… and I shall give it an unoriginal & narcissistic name: Cranky Coin.

After giving it a lot of thought, I decided to use Python. GIL thread concurrency is sufficient. Mining might suffer, but can be replaced with a C mining module. Most importantly, code will be easier to read for open source contributors and will be heavily unit tested. Using frozen pip dependencies, virtualenv, and vagrant or docker, we can fire this up fairly easily under any operating system.

workspace.png

I decided to make Cranky coin a VERY simple, but complete cryptocurrency/blockchain/wallet system. This implementation will not include smart contracts, transaction rewards, nor utilize Merkel trees. Its only purpose is to act as a decentralized ledger. It is rudimentary, but I will eventually fork a few experimental blockchains with advanced features from this one.

The Wallet
This currency will only be compatible with ECC secp256k1 curve key pairs.
We need to provide a mechanism for users to:

– generate key pairs
– get the balance
– create transactions
– sign transactions
– calculate the transaction hash
– broadcast the transaction to the nodes

In order to generate the key pairs, we utilize the pyelliptic library and create a pyelliptic.ECC instance. This will generate a key pair for you. You can also generate an ECC instance with a known key pair.

def __init__(self, private_key=None, public_key=None):
if private_key is not None and public_key is not None:
self.__private_key__ = private_key.decode('hex')
self.__public_key__ = public_key.decode('hex')
self.ecc = self.generate_ecc_instance()
def generate_ecc_instance(self):
if self.__private_key__ is None or self.__public_key__ is None:
print "ECC keys not provided.  Generating ECC keys"
ecc = pyelliptic.ECC(curve='secp256k1')
self.__private_key__ = ecc.get_privkey()
self.__public_key__ = ecc.get_pubkey()
else:
ecc = pyelliptic.ECC(curve='secp256k1', privkey=self.__private_key__, pubkey=self.__public_key__)
return ecc
def get_pubkey(self, hex=True):
return self.ecc.get_pubkey().encode('hex') if hex else self.ecc.get_pubkey()
def get_privkey(self, hex=True):
return self.ecc.get_privkey().encode('hex') if hex else self.ecc.get_privkey()

The main purpose for our private key is to sign transactions originating from our public key. This way, the nodes can validate transactions with the public key. In other words,
signature = sign(priv_key, message)
and
verified = verify(signature, message, pub_key)
or if you prefer,
verify(sign(priv_key, message), message, pub_key) = true
Implemented:

def sign(self, message):
return self.ecc.sign(message).encode('hex')
def verify(self, signature, message, public_key=None):
if public_key is not None:
return pyelliptic.ECC(curve='secp256k1', pubkey=public_key.decode('hex')).verify(signature.decode('hex'), message)
return self.ecc.verify(signature, message)   

In order to check your balance, we’d need to query any full node that has a copy of the entire blockchain. There are better ways of implementing this, but I’ve hacked something together which iterates over each transaction in each block on the chain to search for your public key:

def get_balance(self, address):
balance = 0
for block in self.blocks:
for transaction in block.transactions:
if transaction["from"] == address:
balance -= transaction["amount"]
if transaction["to"] == address:
balance += transaction["amount"]
return balance

In order to spend the coins that belong to our public address, we need to be able to create a transaction, hash it, and sign it with our private key:

def create_transaction(self, to, amount):
timestamp = datetime.datetime.utcnow().isoformat()
signature = self.sign(
self.generate_signable_transaction(
self.get_pubkey(),
to,
amount,
timestamp))
transaction = {
"from": self.get_pubkey(),
"to": to,
"amount": amount,
"signature": signature,
"timestamp": timestamp,
"hash": transaction_hash
}  
transaction["hash"] = self.calculate_transaction_hash(transaction)
return self.broadcast_transaction(transaction)
def calculate_transaction_hash(self, transaction):
"""
Calculates sha-256 hash of transaction
:param transaction: transaction
:type transaction: dict(from, to, amount, timestamp, signature, (hash))
:return: sha256 hash
:rtype: str
"""
# pop hash so method can calculate transactions pre or post hash
data = transaction.copy()
data.pop("hash", None)
data_json = json.dumps(data, sort_keys=True)
hash_object = hashlib.sha256(data_json)
return hash_object.hexdigest()
def generate_signable_transaction(self, from_address, to_address, amount, timestamp):
return ":".join((from_address, to_address, amount, timestamp))   

Now that our transaction is ready, broadcasting it is fairly simple. We first retrieve the latest list of full nodes and then POST the transaction to each node:

def broadcast_transaction(self, transaction):
self.request_nodes_from_all()
bad_nodes = set()
data = {
"transaction": transaction
}
for node in self.full_nodes:
url = TRANSACTIONS_URL.format(node, FULL_NODE_PORT)
try:
response = requests.post(url, data)
except requests.exceptions.RequestException as re:
bad_nodes.add(node)
for node in bad_nodes:
self.remove_node(node)
bad_nodes.clear()
return

Note: Here, I am making requests serially. This is not good and will be slow as the list of nodes grow. It’s very simple to replace “requests” with “grequests” (gevent wrapped requests) which can fire off concurrently and return a list of response objects. By the time you read this post, it may already be changed in the main repo. It’s lame because my explanation took more time than changing a few lines of code.

The Transaction

One may argue that the “transaction” is the smallest unit of a blockchain. Transactions make up the core data on each block.
Our transactions look something like this:

transaction = {
"from": source_public_key,
"to": destination_public_key,
"amount": amount,
"signature": signature,
"timestamp": timestamp,
"hash": transaction_hash
}

When a transaction is verified, the nodes must verify the following:

– source address has enough funds to send the listed amount to the destination address (this is done by looking through the transaction history of the source_public_key)
– signature of the transaction is validated with the public key. (this guarantees that the transaction was created by the owner of the private key)
– the hash of the transaction is correct
– the hash of the transaction is not found anywhere else on the blockchain (prevent a double spend attack)

I’ve reserved the final transaction in each list of transactions for the reward to the miner. This transaction is verified separately…

The Block

When the full node has a list of unconfirmed transactions, it is ready to mine a block. We pop off transactions and validate each one before adding it to our block. When we have enough transactions, we add our reward transaction which must apply the following algorithm:

def get_reward(self, index):
# 50 coins per block.  Halves every 1000 blocks
reward = self.INITIAL_COINS_PER_BLOCK
for i in range(1, ((index / self.HALVING_FREQUENCY) + 1)):
reward = reward / 2
return reward

Initially, the nodes reward 50 coins per block to the miner. Every 1000 blocks mined, this reward is cut in half. Eventually, the number of coins rewarded turns to 0. This is how coins like Bitcoin can guarantee a maximum limit of coins in circulation. In our case, after 6000 blocks have been mined, there will be no more coins released. I haven’t implemented transaction fees into Cranky coin yet, but that is crucial for continued incentivization for mining and keeping the blockchain alive.

Remember: Trying to cheat this will cause our block to get rejected from the other nodes… and our blocks are afraid of rejection. :/

The block is just a data object. (Not really sure if Pythonistas their own term for POJOs :D)

class Block(object):
def __init__(self, index, transactions, previous_hash, current_hash, timestamp, nonce):
"""
:param index: index # of block
:type index: int
:param transactions: list of transactions
:type transactions: list of transaction dicts
:param previous_hash: previous block hash
:type previous_hash: str
:param current_hash: current block hash
:type current_hash: str
:param timestamp: timestamp of block mined
:type timestamp: int
:param nonce: nonce
:type nonce: int
transaction
:type transaction: dict(from, to, amount, timestamp, signature, hash)
:type from: string
:type to: string
:type amount: float
:type timestamp: int
:type signature: string
:type hash: string
"""
self.index = index
self.transactions = transactions
self.previous_hash = previous_hash
self.current_hash = current_hash
self.timestamp = timestamp
self.nonce = nonce
def __repr__(self):
return "<Crankycoin Block {}>".format(self.index)
def __str__(self):
return str(self.__dict__)
def __eq__(self, other):
return self.__dict__ == other.__dict__
def __ne__(self, other):
return not self == other

Note that it contains a list of transactions, current hash, previous hash, and a nonce.

There is one special block, also known as the “genesis block”. This is the first block in the chain and it is hardcoded into the codebase.

def get_genesis_block(self):                                                                                                                    
genesis_transaction = {
"from": "0",
"to": "0409eb9224f408ece7163f40a33274d99b6b3f60e41b447dd45fcc6371f57b88d9d3583c358b1ea8aea4422d17c57de1418554d3a1cd620ca4cb296357888ea596",
"amount": 50,
"signature": "0",
"timestamp": 0,
"hash": 0
}  
genesis_transactions = [genesis_transaction]
current_hash = self.calculate_block_hash(0, 0, 0, genesis_transactions, 0)
genesis_block = Block(0, genesis_transactions, 0, current_hash, 0, 0);
return genesis_block

The Blockchain

In Cranky Coin, the Blockchain is really just an instantiatable class that:

– holds the chain of blocks as a list (Nothing is persisted to disk in this primitive implementation)
– holds the list of unconfirmed transactions (treated like a FIFO queue)
– includes various methods for mining, validating, altering the chain, adding blocks, calculating hashes, and other blockchain tools

This provides the core of Cranky coin’s functionality.

def validate_block(self, block):
# verify genesis block integrity
# TODO implement and use Merkle tree
if block.index == 0:
if block != self.get_genesis_block():
raise GenesisBlockMismatch(block.index, "Genesis Block Mismatch: {}".format(block))
return True
# current hash of data is correct and hash satisfies pattern
if not self._check_hash_and_hash_pattern(block):
raise InvalidHash(block.index, "Invalid Hash: {}".format(block.current_hash))
# block index is correct and previous hash is correct
if not self._check_index_and_previous_hash(block):
raise ChainContinuityError(block.index, "Block not compatible with previous block id: {} and hash: {}".format(block.index-1, block.previous_hash))
# block reward is correct based on block index and halving formula
if not self._check_transactions_and_block_reward(block):
raise InvalidTransactions(block.index, "Transactions not valid.  Insufficient funds and/or incorrect block reward")
return True

In our block validation code, I’ve created and designated just a few different types of exceptions depending on the circumstances behind the validation error.

Our block hash is calculated based on the index, previous hash, timestamp, transactions, and nonce:

def calculate_block_hash(self, index, previous_hash, timestamp, transactions, nonce=0):                                                         
"""
Calculates sha-256 hash of block based on index, previous_hash, timestamp, transactions, and nonce
:param index: index of block to hash
:type index: int
:param previous_hash: previous block hash
:type previous_hash: str
:param timestamp: timestamp of block mined
:type timestamp: int
:param transactions: list of transactions
:type transactions: list of transaction dicts
:param nonce: nonce
:type nonce: int
:return: sha256 hash
:rtype: str
"""
data = {
"index": index,
"previous_hash": previous_hash,
"timestamp": timestamp,
"transactions": transactions,
"nonce": nonce
}  
data_json = json.dumps(data, sort_keys=True)
hash_object = hashlib.sha256(data_json)
return hash_object.hexdigest()

Whenever a new block gets added to our blockchain or chain alteration is in order (we have forked from the main branch), we must go through a series of validations and lock the blocks as the list of blocks is not inherently thread-safe.
For this reason, I establish two locks (blocks and unconfirmed transactions) in the constructor of our blockchain class. I don’t want these to be susceptible to race conditions. I’ve initiate with
self.blocks_lock = threading.Lock()
and
self.unconfirmed_transactions_lock = threading.Lock()

Now, we can modify our blockchain in peace:

def alter_chain(self, blocks):
#TODO enforce finality through key blocks
fork_start = blocks[0].index
alternate_blocks = self.blocks[0:fork_start]
alternate_blocks.extend(blocks)
alternate_chain = Blockchain(alternate_blocks)
if alternate_chain.get_size() > self.get_size():
with self.blocks_lock:
self.blocks = alternate_blocks
return True
return False
def add_block(self, block):
#TODO change this from memory to persistent
if self.validate_block(block):
with self.blocks_lock:
self.blocks.append(block)
return True
return False

When we mine a block, we:

1) pop off transactions from the unconfirmed transactions pool
2) validate them
3) add a reward transaction

def mine_block(self, reward_address):                                                                                                           
#TODO add transaction fees
transactions = []
latest_block = self.get_latest_block()
new_block_id = latest_block.index + 1
previous_hash = latest_block.current_hash
for i in range(0, self.MAX_TRANSACTIONS_PER_BLOCK):
unconfirmed_transaction = self.pop_next_unconfirmed_transaction()
if unconfirmed_transaction is None:
break
if unconfirmed_transaction["hash"] != self.calculate_transaction_hash(unconfirmed_transaction):
continue
if unconfirmed_transaction["hash"] in [transaction["hash"] for transaction in transactions]:
continue
if self.find_duplicate_transactions(unconfirmed_transaction["hash"]):
continue
if not self.verify_signature(
unconfirmed_transaction["signature"],
":".join((
unconfirmed_transaction["from"],
unconfirmed_transaction["to"],
str(unconfirmed_transaction["amount"]),
str(unconfirmed_transaction["timestamp"]))),
unconfirmed_transaction["from"]):
continue
transactions.append(unconfirmed_transaction)
if len(transactions) < 1:
return None
reward_transaction = {
"from": "0",
"to": reward_address,
"amount": self.get_reward(new_block_id),
"signature": "0",
"timestamp": datetime.datetime.utcnow().isoformat()
}
reward_transaction["hash"] = self.calculate_transaction_hash(reward_transaction)
transactions.append(reward_transaction)

Then we:

4) add these transactions to the new block
5) hash the block
6) check to see if the block hash starts with a prefix of “0000”
7) if the block hash doesn’t start with a prefix of “0000”, increase the nonce and go back to step 5.
8) if the block hash does start with a prefix of “0000”, broadcast the block to the other nodes and await confirmation (we are getting ahead of ourselves as interaction with other nodes occurs within our FullNode class)

timestamp = datetime.datetime.utcnow().isoformat()
def new_hash(nonce):
return self.calculate_block_hash(new_block_id, previous_hash, timestamp, transactions, nonce)
i = 0
while new_hash(i)[:4] != "0000":
latest_block = self.get_latest_block()
if latest_block.index >= new_block_id or latest_block.current_hash != previous_hash:
# Next block in sequence was mined by another node.  Stop mining current block.
# identify in-progress transactions that aren't included in the latest_block and place them back in
# the unconfirmed transactions pool
for transaction in transactions[:-1]:
if transaction not in latest_block.transactions:
self.push_unconfirmed_transaction(transaction)
return None
i += 1 
block = Block(new_block_id, transactions, previous_hash, new_hash(i), timestamp, i)
return block

The Node

Our FullNode class is basically the controller to our blockchain. It is our external interface which wraps our blockchain code and is responsible for interacting with other nodes on the Cranky coin network. We shall use Klein as our lightweight Rest API server.

These are the URL constants defined in our node module:

FULL_NODE_PORT = "30013"
NODES_URL = "http://{}:{}/nodes"
TRANSACTIONS_URL = "http://{}:{}/transactions"
BLOCK_URL = "http://{}:{}/block/{}"
BLOCKS_RANGE_URL = "http://{}:{}/blocks/{}/{}"
BLOCKS_URL = "http://{}:{}/blocks"
TRANSACTION_HISTORY_URL = "http://{}:{}/address/{}/transactions"
BALANCE_URL = "http://{}:{}/address/{}/balance"

Furthermore, when we instantiate a full node, the mining loop automatically begins in the background before the API server fires up:

class FullNode(NodeMixin):
NODE_TYPE = "full"
blockchain = None
app = Klein()
def __init__(self, host, reward_address, block_path=None):
self.host = host
self.request_nodes_from_all()
self.reward_address = reward_address
self.broadcast_node(host)
if block_path is None:
self.blockchain = Blockchain()
else:
self.load_blockchain(block_path)
thread = threading.Thread(target=self.mine, args=())
thread.daemon = True
thread.start()
print "\n\nfull node server started...\n\n"
self.app.run("localhost", FULL_NODE_PORT)

The mining method is a wrapper to the blockchain mine method except that

– it loops infinitely
– after a block is mined, it broadcasts it to the other nodes on the network for confirmation before it continues to mine
– if a block is not confirmed, the node re-syncs with the other nodes on the network and recycles the transactions back into the unconfirmed transactions pool

def mine(self):
print "\n\nmining started...\n\n"
while True:
latest_block = self.blockchain.get_latest_block()
latest_hash = latest_block.current_hash
latest_index = latest_block.index
block = self.blockchain.mine_block(self.reward_address)
if not block:
continue
statuses = self.broadcast_block(block)
if statuses['expirations'] > statuses['confirmations'] or \
statuses['invalidations'] > statuses['confirmations']:
self.synchronize()
new_latest_block = self.blockchain.get_latest_block()
if latest_hash != new_latest_block.current_hash or \
latest_index != new_latest_block.index:
#latest_block changed after sync.. don't add the block.
self.blockchain.recycle_transactions(block)
continue
self.blockchain.add_block(block)

When we broadcast a block, we are either expecting a confirmation, invalidation, or expiration.
A confirmation is our happy path scenario where every node mines blocks without any conflicts.
An invalidation occurs when a node reports that either your block or the transactions within are invalid. Ideally, validation checks should not fail as they are validated when mining the block. This usually means foul play or different versions of the node software are running.
An expiration occurs when your block index is behind another node’s. The longest chain (if valid) wins.

def broadcast_block(self, block):
#TODO convert to grequests and concurrently gather a list of responses
statuses = {
"confirmations": 0,
"invalidations": 0,
"expirations": 0
}  
self.request_nodes_from_all()
bad_nodes = set()
data = {
"block": block,
"host": self.host
}  
for node in self.full_nodes:
url = BLOCKS_URL.format(node, FULL_NODE_PORT)
try:
response = requests.post(url, data)
if response.status_code == 202:
# confirmed and accepted by node
statuses["confirmations"] += 1
elif response.status_code == 406:
# invalidated and rejected by node
statuses["invalidations"] += 1
elif response.status_code == 409:
# expired and rejected by node
statuses["expirations"] += 1
except requests.exceptions.RequestException as re:
bad_nodes.add(node)
for node in bad_nodes:
self.remove_node(node)
bad_nodes.clear()
return statuses

Once again, before you criticize me for it, I am broadcasting to the other nodes serially. A simple change to use grequests corrects this.

It is also worth mentioning how my peer-to-peer/peer discovery system works. Rather than relying on the nodes to forward requests/broadcasts to all of the nodes, on cranky coin, every node and client is aware of every node… and thus responsible for broadcasting to everybody.

As long as we start off with at least one good node, we can retrieve a list of all the other nodes, calculate a union of these nodes, and populate or update our own list of nodes.

def request_nodes(self, node, port):
url = NODES_URL.format(node, port)
try:
response = requests.get(url)
if response.status_code == 200:
all_nodes = response.json()
return all_nodes
except requests.exceptions.RequestException as re:
pass
return None
def request_nodes_from_all(self):
full_nodes = self.full_nodes.copy()
bad_nodes = set()
for node in full_nodes:
all_nodes = self.request_nodes(node, FULL_NODE_PORT)
if all_nodes is not None:
full_nodes = full_nodes.union(all_nodes["full_nodes"])
else:
bad_nodes.add(node)
self.full_nodes = full_nodes
for node in bad_nodes:
self.remove_node(node)
return

This mechanism is shared by the light client/wallet as well. Nobody on the network needs to be aware of clients/wallets. However, nodes need to be visible. So when a full node is started, it is also responsible for broadcasting itself to the network:

def broadcast_node(self, host):
self.request_nodes_from_all()
bad_nodes = set()
data = {
"host": host
}  
for node in self.full_nodes:
url = NODES_URL.format(node, FULL_NODE_PORT)
try:
requests.post(url, data)
except requests.exceptions.RequestException as re:
bad_nodes.add(node)
for node in bad_nodes:
self.remove_node(node)
bad_nodes.clear()
return

Here I’ve covered the significant bits and pieces of Cranky coin. The rest can be perused directly from the repository: https://github.com/cranklin/crankycoin
Keep an eye on the repository. I will update the docs with clear instructions on running the tests, running a node, opening a wallet, etc.

Let’s recap:

I’ve walked the reader through the creation of Cranky Coin. Cranky Coin is a primitive yet complete cryptocurrency / blockchain / wallet which relies on PoW (proof-of-work) mining. Similar to Bitcoin classic, all full nodes are mining nodes.

Improvements:

– hash rate should be accounted for and hash difficulty should increase rather than staying static at “0000”.
– merkel trees would speed up transaction validation
– grequests would speed up broadcast requests to all nodes
– persistence of blocks and unconfirmed transactions
– transactions can make room for additional data and/or smart contracts
– unconfirmed_transactions can use a better queueing mechanism vs bastardizing a list as a FIFO
– sharding
– larger blocks with more transactions
– https
– ditch “Proof of Work” for “Proof of ****” (will reveal later)
– better testing (the blockchain module was thoroughly unit tested but the node module was just me brainstorming in code)
– shorter public keys for IBAN compliance
– many cool ideas which I shall reveal later

In case you’re wondering, I am not attempting to ICO Cranky coin. I realize that most ICOs are total garbage or outright scams. This hurts the entire crypto community. Believe it or not, mosts ICOs are raising tens of millions of dollars with a blockchain codebase less mature and less ready than Cranky coin. Some are raising money with nothing more than a “promise” (or whitepaper) to implement a “better blockchain” in the future.

If I ever decide to ICO, I will introduce something with intrinsic value and quality. But at present, the purpose of Cranky coin is to educate and draw developers to the world of cryptocurrency.

BTC donation wallet: 1CrANKo4jXDaromxFJZFJb3dcbu8icFPmJ
ETH donation wallet: 0x649283Bd1eC11eCb51B349867f99c0bEAE96DCf8


report

Recommend

About Joyk


Aggregate valuable and interesting links.
Joyk means Joy of geeK