Blockchain technology mainly includes the following components.
- Cryptographic Hash Functions
We all know that a function can take one or several input values and then produce one or several output values through its operations. A hash function satisfies all the following conditions:
- Accepts strings of any length as input.
- Produces a fixed-length output value.
- Computes within a reasonable time frame.
As long as the above conditions are met, a function can be called a hash function. For example, the modulus operation, where any number modulo 10 results in a number between 0 and 9, can be considered a hash function.
Currently, the hash function used for digital currencies like Bitcoin is a cryptographic hash function. In addition to having the three characteristics of a hash function mentioned above, cryptographic hash functions have more unique properties: collision resistance, hiding, and randomness of results, which will be explained below.
(1) Collision Resistance
Collision resistance is divided into strong collision resistance and weak collision resistance. Strong collision resistance means that for a hash function H, we cannot find two different values x and y such that H(x) = H(y). Weak collision resistance means that for a hash function H and an input value x, we cannot find another value y such that H(x) = H(y).
Collision resistance does not mean there are truly "no" collisions; collisions certainly exist, but the emphasis here is on the difficulty of finding collisions. Taking the hash function SHA256 used in Bitcoin as an example, its output value is 256 bits, so there are only 2^256 possible results, while the input values can be infinite. There is a method that will definitely find a collision: we first find 2^256 + 1 different values, calculate their hash values, and there will definitely be duplicate values in that result set, thus discovering a collision. But how feasible is this method? Suppose we gather all computing devices in the world and have them continuously compute from the moment the universe was born until now; the probability of finding a collision would be the same as the probability of the Earth being struck by an asteroid and destroyed in the next second. Since you have read this far, it means the Earth has not been destroyed, which means no collision has occurred.
Currently, no cryptographic hash function has been mathematically proven to be strictly collision-resistant. The collision resistance mentioned in the market is generally considered to mean that, apart from brute-force attacks, there are no other faster ways to find collisions. There have been cases where hash functions once thought to be collision-resistant later found breaking solutions, such as the MD5 hash algorithm. The SHA256 hash algorithm used in Bitcoin is currently also considered collision-resistant, but it does not rule out the possibility of being broken in the future.
What applications does collision resistance have? A common example is message digests. A message digest refers to the hash value obtained from any length of input through cryptographic hash function operations. Taking the commonly used hash algorithm MD5 as an example, its operation is as follows:
It can be seen that regardless of the length of the input string, the result is always a fixed-length random string. Because of the existence of collision resistance, we can consider this string to uniquely represent the input value.
When we download software from the internet, how can we determine that the software we downloaded is the same as the one on the website? At this time, the message digest can play a role. For example, some websites provide the md5sum value of the software when downloading, so after we download the software, we can manually calculate the md5sum value of the software and compare it with the value on the website. As long as the two values are consistent, it indicates that the software we downloaded is complete and correct.
(2) Hiding
Given H(x), it is impossible to infer the value of x. Not only is it impossible to infer the value of x, but also any characteristics about x, such as parity, etc.
(3) Randomness of Results
Regardless of whether the x values are close, the result H(x) obtained after hashing is completely random.
This characteristic means that even if the input value x is very long, and another input value x' differs from x by only one digit, the results obtained after hashing with function H will have no correlation, just like inputting two completely different x values.
Continuing with the example of md5sum values:
liangpeili@LiangXiaoxin:~$ echo 'aschplatform' | md5sum
150fa3630db1d8f576d1266176f6e0f7 -
liangpeili@LiangXiaoxin:~$ echo 'aschplatform1' | md5sum
e915a617b2301631ec14d1ca2c093c63 -
liangpeili@LiangXiaoxin:~$ echo 'aschplatform2' | md5sum
bbb9d830f4a5d47051f9fd19cb0fc75e -
From the above program, it can be seen that even if only a very small value is changed, the result after hashing will be significantly different.
What is the significance of this feature? If we want to find an input value x that meets a specific result value H(x), there is no other way besides brute-force attempts. Continuing with SHA256 as an example, its output result length is 256 bits. If we want to find such an x value that, after SHA256 computation, the result's first digit is 0, the expected number of attempts to find such an x value is 2. What if we want to obtain a hash value with 10 consecutive zeros? The expected number of attempts would be 2^10. By adjusting the result range, we can adjust the number of computations (which can also be considered as the difficulty of the result), which is also the principle behind Bitcoin's difficulty adjustment.
The mining algorithm implemented in Python is as follows:
#!/usr/bin/env python
#coding:utf-8
example of proof-of-work algorithm
import hashlib
import time
max_nonce = 2 ** 32 # 4 billion
def proof_of_work(header, difficulty_bits):
calculate the difficulty target
target = 2 ** (256-difficulty_bits)
for nonce in xrange(max_nonce):
hash_result = hashlib.sha256(str(header)+str(nonce)).hexdigest()
# check if this is a valid result, below the target
if long(hash_result, 16) < target:
print("Success with nonce %d" % nonce)
print("Hash is %s" % hash_result)
return (hash_result, nonce)
print("Failed after %d (max_nonce) tries" % nonce)
return nonce
if name == "main":
nonce = 0
hash_result = ''
difficult from 0 to 15 bits
for difficulty_bits in xrange(16):
difficulty = 2 ** difficulty_bits
print("Difficulty: %ld (%d bits)" %(difficulty, difficulty_bits))
print("Starting search...")
# checkpoint the current time
start_time = time.time()
# make a new block which includes the hash from the previous block
# we fack a block of transactions - just a string
new_block = 'test block with transactions' + hash_result
# find a valid nonce for the new block
(hash_result, nonce) = proof_of_work(new_block, difficulty_bits)
# checkpoint how long it took to find a result
end_time = time.time()
elapsed_time = end_time - start_time
print("Elapsed Time: %.4f seconds" % elapsed_time)
if elapsed_time > 0:
# estimate the hashes per second
hash_power = float(long(nonce)/elapsed_time)
print("Hashing Power: %ld hashes per second" % hash_power)
2. Digital Signatures
In real work and life, we express our recognition of a document through a signature, which others can recognize and cannot forge. A digital signature is an electronic implementation of a real signature, which not only can fully achieve the characteristics of a real signature but can even do better. Common digital signature algorithms include RSA (Rivest-Shamir-Adleman Scheme), DSS (Digital Signature Standard), etc. Bitcoin uses ECDSA (Elliptic Curve Digital Signature Algorithm) to generate public and private keys for accounts and to verify the digital signatures of transactions and blocks. The working principle is as follows:
1) Alice generates a pair of keys, one is sk (signing key), which is not public; the other is vk (verification key), which is public. This pair of keys is generated simultaneously and is mathematically related, and it is impossible to infer any information about sk from vk.
2) The digital signature algorithm receives two inputs: information M and sk, generating a digital signature Sm.
3) The verification function receives information M, Sm, and vk as inputs, returning a result of yes or no. The purpose of this step is to verify that the digital signature you see for information M is indeed issued by Alice's sk, confirming whether the information matches the signature.
Unlike handwritten signatures, which are generally similar, digital signatures are greatly affected by the input. A slight change in the input will produce a completely different digital signature. Generally, we do not directly sign the information but sign the hash value of the information. Given the collision resistance of cryptographic hash functions, this is as secure as signing the original information.
3. Consensus Mechanisms
A blockchain can be seen as a distributed public ledger that records all transactions, and each node in the blockchain is equal. This raises a question: who has the authority to enter data into this ledger? If several nodes write data to the blockchain simultaneously, whose data will be considered valid? This is the problem of maintaining data consistency in a distributed network. A consensus mechanism refers to a method that allows various nodes participating in the network to reach a consensus on data. In blockchain, the consensus mechanism also includes functions such as block production, block validation, and economic incentives for the system.
Different consensus mechanisms are suitable for different application scenarios. Below are common consensus mechanisms and their applicable scenarios:
Proof of Work (POW) — Bitcoin uses the proof of work consensus mechanism. In this mechanism, any device with computing power can participate in the competition to produce blocks. The system dynamically adjusts the difficulty value based on the current total computing power of the network to ensure that, on average, every 10 minutes, the network will decide which block to recognize based on the attitude towards subsequent blocks. Generally, a transaction is considered relatively secure and irreversible after six confirmations (about one hour). When Satoshi Nakamoto designed Bitcoin, the core idea behind the proof of work mechanism was "one cpu one vote," aiming to make Bitcoin a completely decentralized system where anyone can participate using computers or other terminals. Although later, due to the emergence of mining pools, the computing power of the Bitcoin system became relatively centralized, the proof of work mechanism is still considered the most suitable consensus mechanism for public chains.
- Proof of Stake (POS) — The proof of stake mechanism was proposed in 2013 and was first applied in Peercoin. In the proof of work mechanism, the probability of producing a block is proportional to the computing power you possess. Correspondingly, in the proof of stake mechanism, the difficulty of producing a block is proportional to the stake you hold in the system. In the proof of stake mechanism, the process of producing a block is: nodes use collateral (tokens, assets, reputation, etc., which have value attributes) to bet that a legitimate block will become the new block, and their earnings are the interest on the collateral and transaction service fees. The more collateral provided, the greater the probability of obtaining the right to record. Once a new block is produced, the node can receive corresponding rewards. The goal of the proof of stake mechanism is to solve the problem of massive energy waste in the proof of work mechanism. Malicious participants risk having their collateral confiscated.
- Delegated Proof of Stake (DPOS) — Although both proof of work and proof of stake mechanisms can solve the consistency problem of blockchain data, as mentioned above, the proof of work mechanism has the problem of computing power centralization (mining pools), while the proof of stake mechanism's method of adjusting block production difficulty based on the amount of collateral can lead to the emergence of the "Matthew effect," where accounts with a large number of tokens gain more power and may dominate the right to record. To solve the problems of the first two mechanisms, an improved algorithm based on the proof of stake mechanism was proposed later — the delegated proof of stake mechanism. In this consensus mechanism, every token holder in the system can vote for certain representatives, and the top 101 representatives based on the voting rate can obtain the right to record for the system. These representatives forge blocks at predetermined times and receive rewards for forging blocks. The delegated proof of stake mechanism can improve the efficiency of consensus (compared to Bitcoin, which produces a block every 10 minutes, this mechanism can achieve block production in under 10 seconds) and avoids energy waste and the Matthew effect, making it the choice for many emerging public chains (such as EOS).
4. Transactions in Blockchain
In the Bitcoin network, after each transaction is completed, it is broadcast to the Bitcoin P2P network. Miners can not only receive this transaction but also all other unrecorded transactions within the same time frame. The job of miners is to package all these transactions into a transaction block. The specific process is:
1) Miners will pair these transaction records and calculate the root node value through a Merkle tree.
2) The root node and the hash value of the previous block are combined to form a Challenge String, which miners use as input for proof of work.
3) Miners complete the proof of work and publish the proof for other nodes to verify. At the same time, they allocate mining rewards to themselves in the first record (this record is also called the coinbase transaction).
4) Other nodes verify the block, and if successful, it is added as a new block to the blockchain.
5) Miners can also collect transaction fees from other transaction records for themselves.
The birth of Bitcoin and the continuous development of blockchain technology give us immense imagination. Currently, the internet has completed the transmission of information, while blockchain technology may bring value transmission to the internet. The infrastructure and application scenarios of blockchain technology may still need some time to develop to the current level of internet technology, but the potential of blockchain technology cannot be underestimated.
Developing a blockchain system with 300 lines of code
This section uses Node.js to implement a simple blockchain system in just 300 lines of code.
Creating Blocks and Blockchain
A blockchain is a chain that connects blocks with hash pointers, and blocks are the basic units within it. Here, we start by designing a data structure for a block.
1. Creating a Block
A block is the basic unit that constructs a blockchain, and a block must contain at least the following information:
- index: The position of the block in the blockchain.
- timestamp: The time the block was created.
- transactions: The transactions contained in the block.
- previousHash: The hash value of the previous block.
- hash: The hash value of the current block.
Among these, the last two attributes, previousHash and hash, are the essence of the blockchain, and the immutability feature of the blockchain is guaranteed by these two attributes.
Based on the above information, we create a Block class:
const SHA256 = require('crypto-js/sha256');
class Block {
// Constructor
constructor(index, timestamp) {
this.index = index;
this.timestamp = timestamp;
this.transactions = [];
this.previousHash = '';
this.hash = this.calculateHash();
}
// Calculate the hash value of the block
calculateHash() {
return SHA256(this.index + this.previousHash + this.timestamp + JSON.stringify(this.transactions) + this.nonce).toString();
}
// Add a new transaction to the current block
addNewTransaction(sender, recipient, amount) {
this.transactions.push({
sender,
recipient,
amount
});
}
// View transaction information in the current block
getTransactions() {
return this.transactions;
}
}
In the implementation of the Block class above, we used SHA256 from crypto-js as the hash algorithm for the block, which is also the hash algorithm used in Bitcoin. The transactions are a list of transaction objects, where each transaction is formatted as:
{
sender: sender,
recipient: recipient,
amount: amount
}
Additionally, we added three methods to the Block class: calculateHash, addNewTransaction, and getTransactions, which are used to calculate the current block hash, add new transactions to the current block, and retrieve all transactions in the current block, respectively.
Once the block is constructed, the next step is to consider how to assemble the blocks into a blockchain.
2. Creating a Blockchain
A blockchain is a linked list where each element is a block. The blockchain needs a genesis block to initialize, which is also the first block of the blockchain and needs to be generated manually. When we create the Blockchain class, we need to consider the generation of the genesis block. Below is the code example:
class Blockchain {
constructor() {
this.chain = [this.createGenesisBlock()];
}
// Create the genesis block
createGenesisBlock() {
const genesisBlock = new Block(0, "01/10/2017");
genesisBlock.previousHash = '0';
genesisBlock.addNewTransaction('Leo', 'Janice', 520);
return genesisBlock;
}
// Get the latest block
getLatestBlock() {
return this.chain[this.chain.length - 1];
}
// Add a block to the blockchain
addBlock(newBlock) {
newBlock.previousHash = this.getLatestBlock().hash;
newBlock.hash = newBlock.calculateHash();
this.chain.push(newBlock);
}
// Verify if the current blockchain is valid
isChainValid() {
for (let i = 1; i < this.chain.length; i++){
const currentBlock = this.chain[i];
const previousBlock = this.chain[i - 1];
// Verify if the current block's hash is correct
if(currentBlock.hash !== currentBlock.calculateHash()){
return false;
}
// Verify if the current block's previousHash equals the previous block's hash
if(currentBlock.previousHash !== previousBlock.hash){
return false;
}
}
return true;
}
}
In the Blockchain class, we implemented a method to create the genesis block. Since the genesis block does not have a previous block, previousHash is set to 0. Additionally, we assume that this day is the wedding anniversary of Leo and Janice, and Leo transfers 520 tokens to Janice, thus creating a transaction recorded in the genesis block. Finally, we add this genesis block to the constructor, so the blockchain contains a genesis block. The methods getLatestBlock and addBlock are self-explanatory, meaning to get the latest block and add a new block to the blockchain, respectively. The last method, isChainValid, verifies the validity of the entire blockchain by checking the hash values of the blocks. If the data of the blocks already added to the blockchain is tampered with, this method will return false. We will validate this scenario in the next section.
3. Testing the Blockchain
So far, we have implemented the simplest blockchain. In this section, we will test the created blockchain. The method is to add two complete blocks to the blockchain and demonstrate the immutability feature of the blockchain by attempting to modify the block content.
We first create a blockchain named testCoin. Using the Blockchain class, we create a new object, which should only contain the genesis block:
const testCoin = new Blockchain();
console.log(JSON.stringify(testCoin.chain, undefined, 2));
Running this program will yield the result:
[
{
"index": 0,
"timestamp": "01/10/2017",
"transactions": [
{
"sender": "Leo",
"recipient": "Janice",
"amount": 520
}
],
"previousHash": "0",
"hash": "23975e8996cd37311c7fd0907f9b2511c3bf23cf9c9147cca329dec76d7b544e"
}
]
Now, we create two blocks, each containing a transaction. Then we sequentially add these two blocks to the testCoin blockchain:
block1 = new Block('1', '02/10/2017');
block1.addNewTransaction('Alice', 'Bob', 500);
testCoin.addBlock(block1);
block2 = new Block('2', '03/10/2017');
block2.addNewTransaction('Jack', 'David', 1000);
testCoin.addBlock(block2);
console.log(JSON.stringify(testCoin.chain, undefined, 2));
The result will be:
{
"index": 0,
"timestamp": "01/10/2017",
"transactions": [
{
"sender": "Leo",
"recipient": "Janice",
"amount": 520
}
],
"previousHash": "0",
"hash": "23975e8996cd37311c7fd0907f9b2511c3bf23cf9c9147cca329dec76d7b544e"
},
{
"index": "1",
"timestamp": "02/10/2017",
"transactions": [
{
"sender": "Alice",
"recipient": "Bob",
"amount": 500
}
],
"previousHash": "23975e8996cd37311c7fd0907f9b2511c3bf23cf9c9147cca329dec76d7b544e",
"hash": "32b96fa0bba9a7353e67498d822fb0c1f89c307098295c288459cb44dbc5d0f1"
},
{
"index": "2",
"timestamp": "03/10/2017",
"transactions": [
{
"sender": "Jack",
"recipient": "David",
"amount": 1000
}
],
"previousHash": "32b96fa0bba9a7353e67498d822fb0c1f89c307098295c288459cb44dbc5d0f1",
"hash": "3a0b9a0471bb474f7560968f2f05ff93306cfc26be7f854a36dc4fea92018db2"
}
]
The testCoin now contains three blocks; in addition to the genesis block, the remaining two blocks are the ones we just added. Note whether the previousHash attribute of each block correctly points to the hash value of the previous block.
At this point, we can use the isChainValid method to verify the validity of the blockchain. The return value of console.log(testCoin.isChainValid()) is true.
Where does the immutability of the blockchain manifest? Let's first modify the transaction in the first block. In the first block, Alice transferred 500 to Bob; suppose Alice regrets and only wants to pay 100 to Bob, so she modifies the transaction information as follows:
block1.transactions[0].amount = 100;
console.log(block1.getTransactions());
Alice checks the transaction information on the blockchain and finds it has been changed to 100, feeling relieved. Bob sees this and realizes that the transaction has been tampered with. So Bob starts to collect evidence; how can he prove that the transaction in block1 has been tampered with? Bob can call the isChainValid method to prove that the current testCoin is invalid. Because testCoin.isChainValid() returns false. But why does testCoin.isChainValid() return false? Let's look at the underlying logic: first, Alice modified the transaction content, and at this time, the hash value of block1 must be different from the hash value calculated from the previous transaction. The difference between these two values will trigger isChainValid to return false, which is the function implemented by the following code:
if(currentBlock.hash !== currentBlock.calculateHash()) {
return false;
}
If that’s the case, can’t Alice just modify the hash of block1 while modifying the transaction content? Alice can continue to tamper with the content of other blocks:
block1.transactions[0].amount = 100;
block1.hash = block1.calculateHash();
console.log(testCoin.isChainValid());
In this case, the final result is still false. Why? Because of the following code:
if(currentBlock.previousHash !== previousBlock.hash) {
return false;
}
Each block stores the hash value of the previous block, so modifying just one block is not enough; you also need to modify the previousHash stored in the next block. If we have securely stored the hash value of block2, then no matter what, Alice cannot tamper with existing data without being discovered. In real blockchain projects, modifying one block requires modifying all subsequent blocks, which is also an impossible task. The "hash pointer" feature of the blockchain ensures the immutability of blockchain data.
Proof of Work#
The implemented blockchain system is still relatively simple and does not solve the "double spending" problem that needs to be addressed in electronic currency systems. To maintain the healthy operation of the entire system, it is necessary to design an economic incentive mechanism within the system. In the Bitcoin system, Satoshi Nakamoto designed a "proof of work" mechanism to solve the economic incentive problem and the double spending problem within the system. Below we introduce the principles and implementation of the proof of work algorithm.
- Proof of Work Algorithm
A healthy blockchain system will generate transactions at any time, and we need servers to perform the following tasks: periodically package transactions from a time period (Bitcoin is 10 minutes, Asch is 10 seconds) into a block and add it to the existing blockchain. However, there may be many servers in a blockchain system; how do we determine which server's packaged block is valid? To solve this problem, Bitcoin adopts an algorithm called proof of work to decide which server's packaged block to accept and reward accordingly.
The proof of work algorithm can be simply described as follows: multiple servers simultaneously package transactions from a time period, and after packaging, the block header information is computed using the SHA256 algorithm. There is a variable called nonce in both the block header and the reward transaction (coinbase). If the result does not meet the difficulty requirement (which will be explained later), the nonce value is adjusted, and the computation continues. If a server is the first to compute a block that meets the difficulty requirement, it can broadcast this block. Other servers verify it, and if there are no issues, it can be added to the existing blockchain, and everyone can compete for the next block. This process is also called "mining."
The proof of work algorithm uses the SHA256 hash algorithm, which is characterized by the difficulty of obtaining a specific result through computation, but once a suitable result is computed, it is easy to verify. In the Bitcoin system, finding a block that meets the difficulty requirement takes about 10 minutes, but verifying its validity is instantaneous. We will see this in the code implementation in the next section.
To give a simple example: suppose there is a group of people playing a coin-tossing game, where each person has ten coins, and they take turns tossing all ten coins, finally looking at the order of heads and tails among the ten coins. Since the final result is ordered, there are always 2^10 possible outcomes. Now there is a rule: in one round of the game, whoever first tosses four heads wins a reward. So everyone starts tossing their ten coins and counting the results. There are 26 possibilities for the first four being heads, so the expected number of attempts for one person to achieve that result is 24. If the rule is that the more heads, the more attempts each person will have to make, then as the number of players increases, we can require the first six or eight to be heads, and the time for each round of the game will still be about the same. This is also why Bitcoin's computing power has increased significantly while still maintaining an average of one block every 10 minutes.
Having explained what proof of work is, how do we add it to our blockchain application?
Any data will yield a 256-bit binary value after SHA256 computation. We can adjust the number of leading zeros in the result as the "difficulty value." For example, if we require the last block's SHA256 computation to start with a 0, then on average, every two computations will yield such a result. However, if we require ten consecutive zeros, it will take an average of 2^10 computations to achieve such a result. The system can adjust the number of leading zeros in the computation results to achieve the goal of adjusting difficulty.
We add a nonce variable to the block header. By continuously adjusting the nonce value, we recalculate the entire block's hash until the computed result meets the difficulty requirement.
- Code Implementation of Proof of Work
Based on the concepts from the previous section, we start to modify our current blockchain application. First, we add a nonce variable to the Block class:
class Block {
constructor(index, timestamp) {
this.index = index;
this.timestamp = timestamp;
this.transactions = [];
this.previousHash = '';
this.hash = this.calculateHash();
this.nonce = 0;
}
...
}
Then we add a mineBlock method to the Block class:
mineBlock(difficulty) {
console.log(`Mining block ${this.index}`);
while (this.hash.substring(0, difficulty) !== Array(difficulty + 1).join("0")) {
this.nonce++;
this.hash = this.calculateHash();
}
console.log("BLOCK MINED: " + this.hash);
}
The mineBlock method is used to find the nonce based on the difficulty value; only after finding the appropriate nonce can the block be submitted. Here, difficulty refers to the number of leading zeros in the result. If the computed hash value does not meet the requirement, the nonce is incremented, and the block's hash value is recalculated.
Next, we define a difficulty value in the Blockchain class:
constructor() {
this.chain = [this.createGenesisBlock()];
this.difficulty = 2;
}
We apply the mining process to the process of adding a block to the blockchain:
addBlock(newBlock) {
newBlock.previousHash = this.getLatestBlock().hash;
newBlock.mineBlock(this.difficulty);
this.chain.push(newBlock);
}
At this point, we have completed the modifications to the application. Below we will test the code after adding this part.
We first add just one block:
block1 = new Block('1', '02/10/2017');
block1.addNewTransaction('Alice', 'Bob', 500);
testCoin.addBlock(block1);
console.log(block1);
The operation result will be:
Mining block 1
BLOCK MINED: 005fed00324fcbe1f0ab1703afe94e45a99e197a7df142e669444687f9513e57
Block {
index: '1',
timestamp: '02/10/2017',
transactions: [ { sender: 'Alice', recipient: 'Bob', amount: 500 } ],
previousHash: '31b15cc32d6772f237dcf298d5b7a2417f298f40ce6d8d5fbe07958141df7a4c',
hash: '005fed00324fcbe1f0ab1703afe94e45a99e197a7df142e669444687f9513e57',
nonce: 419
}
Note the nonce value and the hash value. The nonce value indicates the number of computations, and the hash value is the final result. The difficulty value we set this time is 2, so the expected number of computations is 2^8 (since one character in the hash represents 4 bits). What if we change the difficulty value to 3? The operation result will be:
Mining block 1
BLOCK MINED: 000b7f17beaf58bc8fea996a9fed11103ed27ad6d63818b87d89a440cd9757b5
Block {
index: '1',
timestamp: '02/10/2017',
transactions: [ { sender: 'Alice', recipient: 'Bob', amount: 500 } ],
previousHash: '31b15cc32d6772f237dcf298d5b7a2417f298f40ce6d8d5fbe07958141df7a4c',
hash: '000b7f17beaf58bc8fea996a9fed11103ed27ad6d63818b87d89a440cd9757b5',
nonce: 4848
}
As can be seen, the number of computations has increased. As the difficulty value increases, the number of CPU computations will increase exponentially, and the time consumed will also increase.
Providing APIs for Interaction with Blockchain#
- Mining Rewards
Before implementing related APIs, let's first look at what mining rewards are.
The above introduced the principles of mining and implemented the proof of work algorithm, but why are servers willing to contribute their CPU resources to package blocks? The answer is that there is a reward mechanism in mining. After miners package a time period's transactions, they create a new transaction at the first position of the block. This transaction has no sender, and the recipient can be set to anyone (usually set to their own address). How much is the reward? Currently, Bitcoin miners receive a reward of 12.5 BTC for each block they package. This reward transaction is guaranteed by the system and can be verified by any other node.
There are several issues here. First, the issue of reward amount. When Bitcoin was first issued, the reward for each block was 50 BTC, and it has halved every four years; as of July 2018, it is 12.5 BTC. Secondly, can miners create multiple reward transactions or increase the reward amount? Miners can certainly do this, but if they do, the broadcast block will not be verifiable by other nodes. When other nodes receive the block, they will perform legality verification, and if it does not comply with system rules, they will discard the block, and that block will ultimately not be added to the blockchain.
- Code Refactoring
To modify our current code into a form suitable for providing APIs, the following adjustments need to be made:
- Add a currentTransactions property to the Blockchain class to collect the latest transactions and prepare to package them into the next block:
constructor() {
this.chain = [this.createGenesisBlock()];
this.difficulty = 3;
this.currentTransactions = [];
}
-
Move the addNewTransaction method from the Block class to the Blockchain class.
-
Export the Block and Blockchain classes, and rename app.js to blockchain.js.
The final content of blockchain.js should be:
const SHA256 = require('crypto-js/sha256');
// Block class
class Block {
constructor(index, timestamp) {
this.index = index;
this.timestamp = timestamp;
this.transactions = [];
this.previousHash = '';
this.hash = this.calculateHash();
this.nonce = 0;
}
calculateHash() {
return SHA256(this.index + this.previousHash + this.timestamp + JSON.stringify(this.transactions) + this.nonce).toString();
}
mineBlock(difficulty) {
console.log(`Mining block ${this.index}`);
while (this.hash.substring(0, difficulty) !== Array(difficulty + 1).join("0")) {
this.nonce++;
this.hash = this.calculateHash();
}
console.log("BLOCK MINED: " + this.hash);
}
getTransactions() {
return this.transactions;
}
}
// Blockchain class
class Blockchain {
constructor() {
this.chain = [this.createGenesisBlock()];
this.difficulty = 3;
this.currentTransactions = [];
}
addNewTransaction(sender, recipient, amount) {
this.currentTransactions.push({
sender,
recipient,
amount
});
}
createGenesisBlock() {
const genesisBlock = new Block(0, "01/10/2017");
genesisBlock.previousHash = '0';
genesisBlock.transactions.push({
sender: 'Leo',
recipient: 'Janice',
amount: 520
});
return genesisBlock;
}
getLatestBlock() {
return this.chain[this.chain.length - 1];
}
addBlock(newBlock) {
newBlock.previousHash = this.getLatestBlock().hash;
newBlock.mineBlock(this.difficulty);
this.chain.push(newBlock);
}
isChainValid() {
for (let i = 1; i < this.chain.length; i++){
const currentBlock = this.chain[i];
const previousBlock = this.chain[i - 1];
if(currentBlock.hash !== currentBlock.calculateHash()){
return false;
}
if(currentBlock.previousHash !== previousBlock.hash){
return false;
}
}
return true;
}
}
module.exports = {
Block,
Blockchain
}
Note that we also modified the code in the createGenesisBlock method in the Blockchain class.
- Using Express to Provide API Services
To provide API services, we will use the most popular Express framework in Node.js. The blockchain will provide the following three interfaces:
- POST /transactions/new: Add a new transaction in JSON format.
- GET /mine: Package the current transactions into a new block.
- GET /chain: Return the current blockchain.
The basic code is as follows:
const express = require('express');
const uuidv4 = require('uuid/v4');
const Blockchain = require('./blockchain').Blockchain;
const port = process.env.PORT || 3000;
const app = express();
const nodeIdentifier = uuidv4();
const testCoin = new Blockchain();
// API implementation
app.get('/mine', (req, res) => {
res.send("We'll mine a new block.");
});
app.post('/transactions/new', (req, res) => {
res.send("We'll add a new transaction.");
});
app.get('/chain', (req, res) => {
const response = {
chain: testCoin.chain,
length: testCoin.chain.length
}
res.send(response);
})
app.listen(port, () => {
console.log(`Server is up on port ${port}`);
});
Next, we will improve the routes /mine and /transactions/new and add some logging functionality (optional).
First, let's look at the route /transactions/new. In this interface, we receive a transaction in JSON format, as follows:
{
"sender": "my address",
"recipient": "someone else's address",
"amount": 5
}
Then, we add this transaction to the current blockchain's currentTransactions. We will use the body-parser module, and the final code is:
const bodyParser = require("body-parser");
const jsonParser = bodyParser.json();
app.post('/transactions/new', jsonParser, (req, res) => {
const newTransaction = req.body;
testCoin.addNewTransaction(newTransaction);
res.send(`The transaction ${JSON.stringify(newTransaction)} is successfully added to the blockchain.`);
});
Next is the route /mine. The function of this interface is to collect the currently unpackaged transactions, package them into a new block, add a reward transaction (set to 50, recipient address is uuid), perform mining that meets the difficulty requirement, and return the new block information. The code implementation is as follows:
app.get('/mine', (req, res) => {
const latestBlockIndex = testCoin.chain.length;
const newBlock = new Block(latestBlockIndex, new Date().toString());
newBlock.transactions = testCoin.currentTransactions;
// Get a reward for mining the new block
newBlock.transactions.unshift({
sender: '0',
recipient: nodeIdentifier,
amount: 50
});
testCoin.addBlock(newBlock);
testCoin.currentTransactions = [];
res.send(`Mined new block ${JSON.stringify(newBlock, undefined, 2)}`);
});
At this point, the code is basically complete. Finally, we add a logging middleware:
app.use((req, res, next) => {
var now = new Date().toString();
var log = `${now}: ${req.method} ${req.url}`;
console.log(log);
fs.appendFile('server.log', log + '\n', (err) => {
if (err) console.error(err);
});
next();
})
Testing the API
Use Node Server.js to start the application, and we will use Postman to test the current API.
After starting the application, the current blockchain should only have one genesis block. We use /chain to get the current blockchain information.
We can see that the current blockchain only has one block. So how do we add new transactions?
References:
-
Implementing proof-of-work
https://www.savjee.be/2017/09/Implementing-proof-of-work-javascript-blockchain/ -
Learn Blockchains by Building One
https://hackernoon.com/learn-blockchains-by-building-one-117428612f46 -
Building Blockchain in Go
https://jeiwan.cc/posts/building-blockchain-in-go-part-2/ -
Bitcoin whitepaper
https://bitcoin.org/bitcoin.pdf
Smart Contracts#
Through the introduction of the previous two chapters, we have a preliminary understanding of smart contracts and the core concepts of Ethereum. From this chapter onwards, we will gradually introduce how to write smart contracts using Solidity. This chapter will introduce what a contract typically contains. We will discuss from two perspectives: one is the structure of Solidity contract files; the other is the content of the contract.
Solidity File Structure
The source files of Solidity contracts use the extension "sol." From the file structure perspective, a contract file typically contains the following parts: contract version declaration, import of other source files, definition of a contract, and comments, etc.
Contract Version Declaration
The source files of Solidity need to declare a version to inform the compiler of the compiler version supported by this source file. When an incompatible new compiler version appears, it will refuse to compile the old source file. It is a good habit to frequently read the version update logs, especially when major versions are released.
The version declaration is as follows: pragma solidity ^0.4.0: such a source file is incompatible with versions before Solidity 0.4.0 and after Solidity 0.5.0 (the “^” symbol is used to control the second part of the version number). Generally, upgrades to the third part of the version number are just minor changes (no compatibility issues), so this method is usually used instead of specifying a specific version. This way, when the compiler has a bug to fix, there is no need to change the code.
If you want to use a more complex version declaration, then its declaration expression should be consistent with npm, which can be referenced at: https://docs.npmjs.com/misc/semver.
Importing Other Source Files
Solidity supports the import statement, similar to JavaScript (ES6), but Solidity does not have the concept of "default export."
Global import, the import form is as follows:
import "filename";
This imports all global symbols from "filename" (including those imported from other files) into the current global scope.
Custom namespace import, the import form is as follows:
import as symbolName from "filename";
This creates a global namespace symbolName, with members coming from the global symbols of filename.
There is a non-ES6 compatible shorthand syntax that is equivalent:
import {symbol1 as alias,symbol2}from "filename";
Solidity Data Types#
Solidity is a statically typed language. In this chapter, we will delve into Solidity's data types. The main content includes:
- Overview and classification of types
- Boolean type
- Integer type
- Fixed-point type
- Fixed-size byte arrays
- Rational and integer literals
- String literals
- Hexadecimal literals
- Enumerations
- Function types
- Address types
- Address literals
- Data locations
- Arrays
- Structures
- Mappings
- Type conversions
Type Inference Operators
Overview and Classification of Types
Solidity is a statically typed language, and common statically typed languages include C, C++, Java, etc. Statically typed means that the type must be specified for each variable (local or state variable) at compile time (or at least can be inferred). Solidity data types may seem simple to use, but they are also the most prone to vulnerabilities (such as overflow issues).
One point to note is that Solidity types are very concerned about the size of the space they occupy. Additionally, some basic types in Solidity can be combined into complex types. Solidity types are divided into two categories: value types and reference types.
Moreover, different types can also be combined with different operators to support expression operations and execute according to the order of evaluation of expressions.
Value Types
Value types occupy space within 32 bytes, and value type variables always perform value copying when assigned or passed as parameters.
Value types include:
- Boolean type (Boolean)
- Integer type (Integer)
- Fixed-point type (Fixed Point Number)
- Fixed-size byte arrays (Fixed-size Byte Array)
- Rational and integer literals (Rational and Integer Literal)
- String literals (String Literal)
- Hexadecimal literals (Hexadecimal Literal)
- Enumerations (Enum)
- Function types (Function Type)
- Address types (Address)
- Address literals (Address Literal)
Reference Types
Reference types mainly include: arrays (Array), structures (Struct), and mappings (Mapping).
Boolean Type (Boolean)
The boolean type is declared using the bool keyword, as follows:
bool isActive;
bool isOk = false; // with default value
The possible values for the boolean type are the constant values true and false.
The operators supported by the boolean type are as follows:
- !, logical NOT.
- &&, logical AND.
- ||, logical OR.
- ==, equal.
- !=, not equal.
Note: The operators "&&" and "||" are short-circuit operators, such as fx) && gy), when fx) is true, gy) will not be executed; fx) || gy), when fx) is false, gy) will not be executed.
Integer Type (Integer)
Unlike languages like Java, which use short, int, and long to represent integers, Solidity's integers are represented by the keyword int followed by a number indicating the number of bits occupied by the type, which is consistent with the Go language.
The keywords for integers are int8, int16 to int256, with increments of 8.
The corresponding unsigned integers are uint8 to uint256, where uint and int default to uint256 and int256, respectively. The declaration is as follows:
int8 x = -1;
int16 y = 2;
int32 z;
The operators supported by integers are as follows:
- Comparison operators: <=, <, =, !=, >=, > (return boolean values true or false).
- Bitwise operators: &, |, ^ (XOR), ~ (bitwise NOT).
- Arithmetic operators: +, -, unary operator "-", unary operator "+", *, /, % (modulus), ** (exponentiation), << (left shift), >> (right shift).
Notes:
- Integer division is always truncated, but if the operation involves constants (constants will be discussed later), it will not be truncated.
- Division by zero will throw an exception, i.e., x/0 is illegal.
- Right shifting and division are equivalent, i.e., x >> y and x / 2 ** y are equal.
- Left shifting and multiplication are equivalent, i.e., x << y and x * 2 ** y are equal.
- The sign of the result of shift operations depends on the number on the left of the operator. Right shifting a negative number will round down to zero.
- Negative shifting is not allowed, i.e., the number on the right of the operator cannot be negative; otherwise, it will throw a runtime exception, such as 3 >> -1 is illegal.
Let's look at another example:
Operator Shorthand
Similar to C, C++, and Java, some operators support shorthand assignment. For example, a += e is equivalent to a = a + e; similar operators include -=, *=, /=, %=, &=, |=, ^=.
a++ and ++a are equivalent to a += 1 and a = 1, but the expression still evaluates to the value of a. a-- and --a return the value after the change.
Integer Overflow Issues
When using integers, special attention should be paid to the size of the integer and the maximum and minimum values it can hold. Many contracts have vulnerabilities due to overflow issues, such as the vulnerability in the Ethereum chain; you can refer to the blog article (https://learnblockchain.cn/2018/04/25bec-overflow/) for more information.
Here are a few examples related to overflow issues:
One way to avoid overflow is to check the result value after the operation, such as checking k after the operation, like using assert(k >= i). It is also recommended to use OpenZeppelin's SafeMath library for addition, subtraction, multiplication, and division operations. The GitHub address for the code is https://github.com/OpenZeppelin/openzeppelin-solidity/blob/master/contracts/math/SafeMath.sol.
Fixed-Point Type (Fixed Point Number)#
The fixed-point type functions similarly to the float and double types in other languages, which are used to represent floating-point numbers. However, the fixed-point type has a unique aspect: it requires specifying the size occupied by the type and the number of decimal places at declaration, while traditional float and double types usually have different values, meaning they occupy different spaces. Solidity does not fully support fixed-point types; they can be declared but not assigned values, so they cannot currently be used.
The declaration of fixed-point types is as follows:
ufixed32x1 f;
ufixed32x1 fi = 0.1; // UnimplementedFeatureError: Not yet implemented
fixed/ufixed represents signed and unsigned fixed-point numbers. The keywords are ufixedMxN and fixedMxN, where M represents the number of bits occupied by this type, in increments of 8, ranging from 8 to 256 bits; N represents the number of decimal places, which can be from 0 to 80. fixed/ufixed respectively represent fixed128x18 and ufixed128x18.
The operators supported are:
- Comparison operators: <=, <, =, !=, >=, > (return boolean values true or false).
- Arithmetic operators: +, -, unary operator "-", unary operator "+", *, /, % (modulus).
Note: Unlike most languages' float and double types, here M represents the total number of bits occupied by the entire number, including both the integer and decimal parts. Therefore, when using a small bit size (small M) to represent a floating-point number, the decimal part will almost occupy the entire space.
Fixed-Size Byte Arrays (Fixed-size Byte Array)#
A fixed-size byte array refers to an array with a fixed space, where each element is a byte. Since variable-length arrays are not value types, we will introduce them in a separate section.
The declaration of fixed-size byte arrays is as follows:
byte bte;
bytes1 bt1 = 0x01;
bytes2 bt2 = "ab";
bytes3 bt3 = "abc";
The keywords are bytes1, bytes2, bytes3, ..., bytes32, incrementing by 1. byte represents bytes1. In practical use, fixed-size byte arrays are often used to replace strings (as in the declarations of bt2 and bt3). The operators supported by fixed-size byte arrays are:
- Comparison operators: <=, <, =, !=, >=, > (return boolean values true or false).
- Bitwise operators: &, |, ^ (bitwise XOR), ~ (bitwise NOT), << (left shift), >> (right shift).
- Index (subscript) access: If x is bytes1, when 0 ≤ k < I, then x[k] returns the k-th byte (read-only). The results of shift operations are similar to integers; the sign of the result depends on the number on the left of the operator, and negative shifting is not allowed. For example, 5 << -1 is illegal.
Fixed-size byte arrays have a member variable: length, which indicates the length of this byte array (cannot be modified).
For example, to get the lengths of bt2 and bt3:
bt2.length; // returns 2
bt3.length; // returns 3
Rational and Integer Literals
Rational and integer literals are numeric constants that appear directly in expressions, but they are also referred to as literals.
Integer literals consist of a series of digits from 0 to 9, represented in decimal. For example, octal numbers do not exist; leading zeros are invalid in Solidity. Decimal fraction literals contain a ".", with at least one digit on either side; valid representations include 1, 1.0, and 13. It also supports scientific notation, where the base can be a decimal, and the exponent must be an integer; valid representations include 2e10, -2e10, 2e-10, and 2.5e1. The numeric constant expressions themselves support arbitrary precision, meaning there will be no overflow or division truncation. However, when converted to corresponding non-constant types or when operated with non-constants, precision cannot be guaranteed. For example, (2 * 800 + 1) - 2 ** 800 results in 1 (uint8 type, even though the intermediate result has exceeded the computer's word length): 5 * 8 results in 4 (even though a non-integer number is involved in the operation): 5 / 2 + 52 results in 5 (will convert to rational number and will not be truncated, although in earlier versions it would have been truncated). As long as the operands are integers, the operators supported by integers apply to integer literal expressions. If both operands are decimals, bitwise operations are not allowed, and the exponent cannot be a decimal.
Example code is as follows:
pragma solidity ^0.4.18;
contract testType2 {
function interLiteral() public returns (uint, uint) {
return (2800 + 1) - 2880, 0.5 * 8;
}
}
Note: Every numeric constant in Solidity has a corresponding numeric constant type. Integer constants and rational constants belong to the numeric constant type. The results of all numeric constant expressions are numeric constants. In numeric constant expressions, once a non-constant expression is included, it will be converted to a non-constant type.