Understanding distributed systems infrastructure

Since the inception of Bitcoin and subsequent innovations in the area of blockchain technologies, there has been a renewed interest in distributed systems and peer-to-peer networks. Although blockchain networks are a relatively new paradigm, the foundations upon which they are based date as far as the 1960s with the emergence of early computer networks.

To understand blockchain technology, it helps to have a basic understanding of the fields of computer science from which they emerge, distributed computing. This article focuses on classifying blockchain networks in the broader field of study known as distributed systems, and differentiates blockchain technologies from digital ledger technologies.

Distributed computing

The history of distributed computing is closely linked to that of the Internet and the primitive networks which preceded it. As computer networks began to emerge in the 1960s, so did the notion that they could act in unison to achieve common goals became apparent. An early example of a distributed system was ARPANET, the precursor to the modern Internet, and later ARPANET e-mail, which came online in the early 1970s.

Distributed systems are made of several independent and interconnected computational units (nodes), each having their own memory. Nodes communicate with each other by passing messages under a common protocol. Such protocols are sometimes referred to as gossip networks.

Nodes in distributed systems work in coordination to achieve a common goal. In many cases, the goal is to solve large computational problems which would otherwise be inaccessible to a single computer. However, they possess other qualities beyond the superior computational abilities of the system.

1. Fault tolerance: the system may continue to operate if individual nodes fail, up to a certain threshold.
2. Partial knowledge: each node only has access to a part of the data processed by the system in aggregate. 

Examples of distributed systems include telecommunications networks, such as cellular networks, network applications, such as the World Wide Web and Bitcoin, industrial systems, such as air traffic control systems, and parallel computing networks such as distributed video rendering and the Human Genome Project.

Distributed ledgers

A distributed ledger is a system of replicated, shared, and synchronized digital data geographically spread across multiple sites, countries, or institutions. A peer-to-peer network of nodes (participants) takes part in a consensus protocol ensuring data replication across all nodes.

Often referred to as Distributed Ledger Technology (DLT), this term can be confusing. Indeed, they do not necessarily represent a single technology, but rather many concepts and technologies, which one may deploy as a system. There are several vendors and open-source projects which offer DLT software specializing in a wide range of applications. 

Trust assumptions

Distributed ledgers operate under an adversarial trust model in which other nodes cannot be trusted. Participants in a distributed ledger network assume other participants cannot be trusted, yet they cooperate to present a single source of truth to external users. There is no central authority or administrator in a distributed ledger network.

This is in contrast with distributed databases, which have existed in production applications for several decades. Although they share many similarities with distributed ledgers, there is one aspect which generally sets them apart: trust between participants. Indeed, participants in a distributed database operate under the assumption that other nodes can be trusted. This is why distributed databases are commonly deployed and controlled by a single administrator.

Properties of distributed ledgers

Digital ledgers possess three main properties.

Append only
Compared to traditional databases which operate under the CRUD model (create, read, update, delete), distributed ledgers are append-only. This means users may only read and add records – records may neither be changed nor removed. 

Shared
The ledger is accessible and maintained by a group of participants which hold a common interest in having accurate data. 

Distributed
The data is duplicated and replicated across participants’ nodes, making it resistant to failure and censorship. 

These properties make distributed ledgers highly resilient, and as an artefact, immutable. 

Examples of distributed ledger applications

• Hashgraph
• Directed Acyclic Graph 
• Blockchain

A blockchain is a type of distributed ledger where data is packaged in a growing list of records called blocks which reference each other using cryptography. Each block contains a cryptographic fingerprint (hash) of the previous block, a timestamp, and raw data.

Blockchains inherit from the distributed ledger concept, in that their data is a) replicated, b) shared between participants, and c) synchronized between participants. Blockchains generally allow for the possibility of generating consensus allowing participants to establish a single source of truth about the integrity of the data. In theory, a blockchain protocol can have a single participant and thus lose its distributed characteristic. Although in practice, there is little use for this configuration, if only in the initial phases of a blockchain network’s lifespan where the goal is to add participants subsequently. 

Blockchain networks are well suited for applications where stakeholders require a) timestamped data, b) the record of its mutation over time, and c) preservation and assurance of the integrity of said data’s state. In practice, the most common applications stem from finance.  

Blockchain technologies and financial applications 

Financial markets require data integrity and symmetry of information between participants. These assurances are today ensured by regulatory oversight of market actors where government mandate enforces a level playing field for market participants and smooth functioning of markets. In traditional market structures, participants have relied on infrastructure, information, and services provided by private entities to provide a trusted market environment.

Blockchain-based financial applications shift some forms of trust from authorities and regulated entities to cryptoeconomic protocols. Such applications exist on peer-to-peer networks composed of anonymous or semi-anonymous stakeholders. These networks are typically referred to as “public” blockchain networks, insofar as they operate on the public Internet and enable anyone to interact with the protocol. 

Examples where trust is shifted

• Compound money market solvency: solvency can be verified and assured by the blockchain network without the need for an authority to maintain system solvency. 
• Transaction settlement: transactions are settled and verified by the blockchain network without the need for an authority or regulated intermediary to verify its validity.

Examples where trust is not shifted

• Prevention of insider information-based trading: the role of authorities stays the same as the blockchain network cannot ensure level playing field vis-a-vis insider information by default.

Example where authorities have a potentially greater role to play

• Systemic risk in blockchain protocols, where failure of one protocol can lead to cascading effects on other protocols leading to a collapse of market activity

Cryptoeconomic protocols reduce reliance on intermediaries by aligning economic incentives. In the case of blockchain networks, those incentives motivate participants to provide the timestamped, provably reliable, and symmetric data needed for markets to function. Indeed blockchain technology creates a new base layer for various forms of markets to emerge, and exciting financial instruments to flourish. 

Understanding the “open-closed” blockchain spectrum

Since the emergence of blockchain networks which permit the programmatic execution of transactions (e.i., Ethereum), there has been some discussion about so-called “public” and “private” blockchains. There may also be referred to colloquially as “permissionless“ and “permissionned“ networks, respectively. 

The distinction between private and public blockchain networks is based solely on their implementation – the underlying technology stack has no means of understanding its own configuration. Indeed, it is possible to configure a blockchain network so that access may be more, or less, open.

It is more suitable to speak of open or closed systems operating on blockchain networks. As we will see, there are nuances which may make a blockchain network more or less open. The term, “open” is better suited, just as a door which may be open at a multitude of degrees, blockchain networks may be configured in any number of ways which creates a gradient on the open-closed spectrum. This is in contrast to the often-used public and private terminology which leaves no daylight between these extremes. 

Note: One should not confuse open access with the concept of open-source software. Although the overwhelming majority of blockchain protocols are indeed open-sourced, this makes reference to a different idea. 

Open refers to the ability for any legal person to leverage the blockchain to conduct business. An open blockchain network is a self-reliant platform on which economic activity can occur, meaning that anyone can participate in the market by validating transactions, developing applications which provide value, use services, or transact. Indeed, open blockchain networks are economic platforms where market rules are enforced by the protocol. 

Closed blockchain networks typically serve the interests of a group of participants who wish to collaborate or do business around a niche use case or application. They are purpose-built and access is restricted to a small number of interested players. 

Although fully closed, or “permissioned” blockchain networks did appear to address a subset of applications in the technology’s early development phase, advances in scalability, privacy and security are making it possible for most applications to exist on open networks. This shift is analogous to how corporate intranets, common in the nineties and early aughts, moved to the cloud in the last decade. Similarly, as technologies progress and gain acceptance, open blockchain networks will capture most applications. Furthermore, just as with Internet infrastructure, it is possible to build “closed” applications on open blockchain networks.

Note: “Private“ or “permissioned“ networks should not be confused with distributed ledger technologies, as described earlier. Indeed, a blockchain is a form of DLT. However, a DLT is not a blockchain. A DLT can exist that does not embody all the properties of a blockchain, regardless of its configuration. 

In summary, the idea of open blockchain networks and closed ones are not in strict opposition to one another. There are nuances and gradients of permissioned access, making it difficult to classify each of these concepts in well-defined categories. 

Network participants’ roles

When interacting with a blockchain network, there are three broad roles a participant may embody:

• application user, such as sending and receiving transactions, and interacting with applications;
• application developer, such as developing and deploying smart contracts, and;
• network validator, such as approving blocks by participating in consensus (e.g., mining, staking).

These roles may overlap as participants may assume different functions (e.g., block validation activities require sending and receiving transactions).

While typically, blockchains networks either allow full open access to all three of these roles (so-called “public” blockchain networks) or restrict access to authorised entities (so-called “private” blockchain networks), there exist practical and theoretical examples where the public/private classification is not so clear. 

The backbone of the Internet is built on open protocols and standards (e.g., TCP/IP, HTTP, SMTP), and open networks operated by private enterprise. Similarly, open blockchain networks can support production applications built by private enterprises with universal access.

While blockchains networks typically allow full open access to all three of these functions or restrict access to private actors, there are practical and theoretical examples where the open/closed classification is not clear. 

To illustrate some instances where the open vs. closed classification is not so clear, we can contrast three different systems in implementation, all potentially useful for the purpose of issuing securities on a blockchain network:

• The ERC20 token standard. It allows an issuer to issue securities on the Ethereum mainnet, a public blockchain network with open validation. ERC20 tokens can be transferred between any pairs of users. No identifying information is required prior to the submission of transactions. 
• The ST20 token standard. This allows an issuer to issue securities on the Ethereum mainnet, a public blockchain network with open validation. ST20 token transfers require the sending and receiving accounts to have their identity registered with the issuer. The issuer, can additionally, create policies for the approval or rejection of ST20 token transfers. While the underlying infrastructure is “open”, the ST20 standard running on top restricts open access. 
• A closed Ethereum network. This allows the issuer to create a blockchain network which is purpose-built for issuing securities. The blockchain’s underlying infrastructure is closed and validation reserved to identified entities.

Conclusion

In conclusion, the emergence of blockchain technologies has brought an enormous amount of attention and research in the fields of peer-to-peer systems, cryptography, game theory and the areas where they overlap. In the last 10 years, new and exciting distributed systems have emerged which serve practical applications in crypto-finance. 

In this context, a clear understanding of these technologies, their unique characteristics and their applications is important. As blockchain networks and applications grow and become regulated, it is in the interest of policy makers to be precise such that private enterprise may continue to innovate and consumers be protected.