# General Architecture
# Abstract
This document describes Miyabi, a blockchain solution for high-speed decentralized applications. It is a world-state-based blockchain as opposed to a UTXO-based blockchain. It can be seen as a state machine that executes transactions in each block and updates the state accordingly. Miyabi allows a rich form of data to be stored and handled.
Miyabi is a permissioned consortium-based permissioned blockchain. Only known nodes and members of the consortium with the required credentials can join a particular Miyabi network. In a Miyabi network, nodes running the consensus algorithm are called consensus nodes. In Miyabi, consensus nodes use the BFK2 consensus algorithm to agree on the content and order of transactions and their execution results within blocks (form a consensus). BFK2 achieves Byzantine Fault Tolerance (BFT) and guarantees the finality of transactions’ finality. Since no more than
Miyabi also supports smart contracts to support customized logics. Miyabi guarantees that the time-stamped contracts are tamper-proof. To prevent potential bugs in smart contracts, Miyabi has two layers of validations. In the first layer, a smart-contract analyzer and a compiler checks for potential syntax issues. In the second layer, each data set (assets, etc.) has independent validation process to check if the operation conforms to a set of predefined rules. For example, we have a special asset validator named Kotowari (理) which checks special validation rule over the touched asset data, ensuring that the remaining balance after an asset move does not fall below a minimum value among other checks.
# Distributed State Machine
Miyabi can be seen as a distributed state machine where each node participates to decide the next state. A transaction can be viewed as an operation on a given state S of the blockchain. A transaction has the capability of making the state machine to transition from state S to S'. Miyabi satisfies the following properties:
- Shares the same initial state
- Shares the same transaction sets that are executed in the same order
- All nodes can be distributed physically without violating consistency
The nodes in Miyabi start with the same initial state. As the peer to peer network receives the transactions, the transactions are bundled into blocks and are executed. New blocks are subsequently added to the blockchain. The order of transactions in a block and the execution result of the transactions are guaranteed to be consistent in accordance with BFK2's assumptions. Since the consistency of the blockchain is guaranteed by BFK2, the order of transactions for each node is consistent. Finally, deterministic transaction execution is implemented. For a given world state and transaction lists, all nodes will produce the same result. The structure of the state machine is shown in Figure 1.
 |
|---|
| Figure 1. The concept of a state machine |
# Node
A node is an instance running the Miyabi program. It has the following major functions:
- Receiving transactions
- Sharing information with other peer nodes (including transactions and blocks)
- Executing transactions and updating world state
- Exposing public and admin APIs
- Run consensus algorithm (consensus nodes only)
The nodes expose a set of public APIs which the users can use to get information related to:
- Metadata of Miyabi
- World state
- Blocks
- Transactions
Additionally, users can also send transactions to a node through its API.
After a simple validation, the node stores the received transaction in an internal transaction memory storage (memory pool). A transaction memory pool is an in-memory data structure which holds the transactions received from the clients which have not been picked up by nodes for generating a new block yet. A transaction is added to the memory pool after verifying the signatures on it. If the signature verification fails, the transaction is dropped. Each node uses an inventory protocol to share newly received transactions and blocks between nodes. The inventory protocol guarantees the sent message will be received eventually. The memory pool of different nodes does not need to be consistent with the transactions they hold. In fact, the inconsistency is natural because of the asynchronous network. The inventory protocol makes sure that the node has a particular transaction it needs in case it is not present in the memory pool. The framework of the inventory protocol is shown in Figure 2.
 |
|---|
| Figure 2. Tx sharing among nodes using inventory protocol |
To generate a block, one of the consensus nodes picks up a set of transactions from the memory pool, executes them, and creates a block "proposal". If other nodes agree on this proposal by the mechanism of consensus, then the proposal is accepted, the block is added by the network, and the state is updated. Due to the asynchronous nature of the network, the order of received transactions and the transactions included in the block are not guaranteed to be consistent. However, the order of the transactions within a committed block is guaranteed to be consistent across the network. If the transaction is out of date, it will be dropped from the memory pool, hence they will not be entered into any block. There can be multiple candidates for the next block. This mechanism is controlled by the consensus layer of Miyabi. However, at the end of the consensus process for the new block, the network commits the same block in accordance with the finality guaranteed by the BFK2 consensus algorithm. A sketch of BFK2 is shown in Figure 3.
 |
|---|
| Figure 3. Illustration of consensus algorithm |
Committed blocks will be transferred to all nodes through inventory protocol. Upon receiving a new block, the node will execute all the transactions within it and update the key-value based tables within the world state.
Each node possesses a unique node ID, which is the public key of the node that is shared by other nodes. The private key related to the node ID is stored as a PKCS#12 file whose path is recorded in the config file of the node. Each private key should only be accessed by the node, and it is used for signing the message to prove its integrity and reliability.
Apart from consensus nodes, Miyabi also has a special set of nodes called application nodes. The application nodes do not participate in the consensus mechanism. The application nodes add a block locally only after the consensus nodes have committed the block to the blockchain. Apart from the consensus mechanism, the application nodes run all the protocols which the consensus node runs. Additionally, application nodes also have a memory pool and transaction resending protocol for relaying the requests of clients. Application nodes can improve the capability of Miyabi without raising the complexity of consensus building. Application nodes can access the geographically closest nodes to reduce latency as shown in Figure 4. The set of all consensus nodes is called the consensus members. It is possible to change the consensus members by transactions that add or remove specific node IDs from the list. Hence, an application node can become a consensus node if its ID is added to the consensus members.
 |
|---|
| Figure 4. Consensus nodes and application nodes |
# Consensus Algorithm - BFK2
Miyabi uses a specially crafted consensus algorithm called BFK2. BFK2 has been internally developed while considering various needs and use cases of Miyabi. Miyabi takes inspiration from the already existing consensus algorithms of public blockchains while removing the various bottlenecks that they have. BFK2 achieves the following attributes:
- Security: Ensures that the consensus can not be reached on an invalid proposal
- Liveness: Ensures that the consensus will eventually be reached
- Finality: Achieved consensus cannot be reversed or revised
BFK2 guarantees the attributes above when Byzantine consensus nodes are less than
BFK2 is a communication-based consensus algorithm, so it can achieve consensus within seconds with minimal network latency. BFK2 can also survive a limited attack on the Byzantine nodes. With N nodes participating in the consensus, the algorithm guarantees security if the number of byzantine nodes is less than N/3, and liveness if the total number of faulty nodes is less than N/3. The term Quorum refers to any sufficiently large subset of validator nodes. An agreement among a Quorum offers a strong guarantee for the progress of the consensus algorithm. Assume a Miyabi has
If
In this situation,
# Structure of World State
The Miyabi blockchain, as stated earlier, manages a distributed state machine. The state of the Miyabi blockchain, which is common among all the nodes (unless they lag behind due to certain situations), is called the "world state." The fundamental structure of the world state is a set of tables. A table represents a logical group of data, which can have independent ownership rights, data types, and read and write rules. Each table is backed by an isolated binary key-value store. In addition, users can use a smart contract to define the logic of modifying their data for complex operations. Miyabi provides a Roslyn analyzer for security checks of smart contacts.
Users can only update the world state by transactions. Miyabi core defines some operations to modify the configuration of world states and some table configurations. Miyabi uses modules to implement different business logic including operation validations and permission check. All tables also support a track feature which can log the transactions that update the table’s data for traceability. Miyabi uses a hash tree to verify the existence of a table or a table’s data. Miyabi calculates the hash tree for the entries in a table with support proof flag. By using the root hash of all table trees, Miyabi can calculate the hash tree for the whole world state and provide a route to the specific entry as a proof of the existence of that entry. All users can calculate the entry and the proof to get the root hash which should be equal to the one in the specific block header for being accepted.
# Structure of Blocks
A block is a collection of transactions along with other relevant information. These blocks are linked together to form an immutable blockchain. A block is included into the blockchain if the Quorum of nodes reach a consensus to include it. Blocks can also be denoted by a unique integer, height, which is the position of this block in the blockchain.
Each block contains three parts: block header, block body, and block evidence (Figure 5).
The block header is a data structure which encapsulates information about the previous block, the events in the current block, and a Merkle root hash of the current block’s transactions and transaction results.
The block body is a data structure encapsulating information about all the transactions and the respective transaction results. It also contains information regarding the events which were triggered during the execution of the transactions in the block.
The block evidence is a data structure that proves the acceptance of a block by the consensus nodes.
 |
|---|
| Figure 5. Structure of Block |
The hash of the block header of a block is referred to as block ID of that block. The previous block's block ID is also stored in the block header. Hence, if one committed block is changed, the ID of that block will be changed which will affect all the IDs of the following blocks which will be detected. This means that the integrity of the blockchain is guaranteed. The structure of blockchain is shown in Figure 6.
 |
|---|
| Figure 6. Blockchain Structure |
# Transaction Structure
Transactions have the structure shown in Figure 7.
 |
|---|
| Figure 7. Transaction Structure |
The transaction header includes the metadata of the transaction, including the time it is created and the expiration time.
Transaction entry is the minimum unit of execution in Miyabi. One transaction can have multiple entries.
Required credentials are the public keys needed. When the transaction is executed and the related permission is required, the node will try to compare the permissioned keys and the required credentials. If any required permission is not satisfied, the transaction will fail.
The signatures for transaction are the signatures for the transaction ID which should be signed by those designated in the required credentials to guarantee the transaction’s integrity. If any signature is missing or incorrect, the transaction will be rejected.
There will be a format check for each transaction. If the transaction does not pass the format check, it will not be recognized as a valid transaction. Format checks include:
- The order of signatures for transaction is equal to the related key's order in required credentials;
- The public key in required credentials and signatures for transactions are matched according to the used cryptography algorithms.
# Execution of Transaction
All nodes in Miyabi will execute transactions according to their order in the block. If a transaction contains multiple entries, they will be executed according to the order within the transaction. Hence, all entries in a block will be executed with the same order.
Execution of entry means this entry will be deserialized and executed according to the specific rule to read and write the current world state of Miyabi. If an entry is failed, the whole transaction will be failed and all the changes to the world state will be rolled back. Only the successful transaction, that all entries within it are successful, can update the world state. The following table provides the possible status of transactions.
| Code | Description | Comment |
|---|---|---|
| -1 | Pending | The transaction is in the mempool and is waiting to be processed |
| 0 | Success | The transaction completed successfully, and any state changes will be reflected after the corresponding block |
| 1 | Negative Balance | (Asset-specific) Negative-balance rule was violated |
| 2 | Conservation Break | (Asset-specific) Total asset-value conservation rule was violated |
| 3 | Failed | Transaction failed for an unspecified reason |
| 4 | Canceled | Transaction was canceled |
| 5 | Signature Invalid | The transaction was not accepted because its signature was not valid and/or did not match the required credentials |
| 6 | Invalid Credentials | Credentials were not valid for this operation |
| 7 | Invalid Operation | The transaction performed an invalid operation while executing |
| 9 | Duplicate | A duplicate transaction was already in the mempool or a committed block |
| 10 | Corrupted | The transaction was not formatted correctly |
| 11 | Invalid Lifetime | The lifetime of the transaction was invalid |
| 12 | No Vacancy | Memory pool is full |
| 13 | Time Out | (Contract-specific) Execution timed out |
# Event
Block event is a specific area in the Block Body used to store plain data. Event contains the block id and a set of transaction events. Each transaction event contains a filter and a set of triggered events. For simplification, we use the event to denote the triggered event. Event is used for data that are generated during the execution of transactions and will not be stored in a world state. Currently, Miyabi support two ways to trigger events: using some specific transaction entry and smart contract.
Below are some example entries that will create events: ConsensusChangeEntry will store the new credentials and the height it will take effect. BlockchainParametersChange will store the activation height, new blockchain config, and its hash to the event. UpdateWorldPermission and UpdateContractPermission will create multiple events to store the touched addresses with their new permissions. This event will be automatically stored in the block where this entry is committed.
Users can get events from block through API GetBlockEvents and use BlockEvent.Events.Any(a => a.Maycontain(filter)) through SDK to judge if the block contains the specific events. The filter is generated by combing all target FilterDatas in an empty EventFilterData.
# Access Control Model
The access control model of Miyabi is based on asymmetric cryptography. The world state and some tables that need access control will independent store addresses for their permissions. When one of those permission is required for an entry, the node will check whether the required credentials of the transaction can verify an address with that permission. These relationships are shown in Figure 8. An address can either be a public key, a threshold public key, or a specific smart contract address. If the permission is granted to a smart contract, all methods in the smart contract can pass the permission check.
 |
|---|
| Figure 8. Access Control Model |
# Executed Transactions
Once a transaction is executed, it will be stored for searching. Each node will maintain a cache for recently executed transactions while other transactions are serialized and stored within local storage. All transaction IDs will be stored if they are not dropped from the memory pool. If the same transaction is sent twice, a transaction filter will detect it and prevent the same transaction from being executed again. To improve efficiency, the filter only stores the transactions executed within a fixed period. If the lifetime of a transaction is longer than the fixed period, the transaction will be rejected directly with an invalid lifetime.
# Digital Signature
A digital signature is an application of asymmetric cryptography. A digital signature service should support the following functions:
- Create key pairs
- Create a signature of a plaintext by private key
- Verify each signature by public key
The workflow of a digital signature service is shown in Figure 9.
 |
|---|
| Figure 9. Digital Signature |
# Access to Miyabi
As previously stated, all Miyabi nodes provide a web API service to the public which is different from the port used for communication between nodes. A client usually does not use web API directly, but accesses the node through the client SDK. The client SDK has wrapped the configuration of web API and provides user-friendly functions and services. By using the client SDK, a client can easily get information from nodes, create signatures, create transactions, and send transactions.
Miyabi also provides a CLI program that wraps client SDK to provide CLI service. Their dependencies are shown in Figure 10.
 |
|---|
| Figure 10. Access of Miyabi |