Sharding is a method of separating and storing a single logical data set across multiple databases. Why do we need it and how does it work? Let’s read together!
Horizontal data separation is another definition of sharding.
Since the late 1990s, the concept of sharding has been used in the management of traditional centralized databases. The term “shard” (fragment) became popular thanks to Ultima Online, one of the first multiplayer online role-playing games, in which developers distributed players across different servers (different “worlds” in the game) to deal with traffic.
The division of the user database by geographical locations is a popular scenario for using sharding in business. Users from the same geographical location are grouped together and assigned to a single server.
Blockchain is a database made up of nodes that represent individual servers. Sharding in the context of blockchain refers to the division of the blockchain network into individual segments (shards). Every shard has its own set of smart contracts and account balances.
Unlike the scheme in which each node is responsible for verifying every transaction in the entire network, each ball is assigned a node that verifies transactions and operations.
By dividing the blockchain into more manageable segments, you can increase transaction throughput and thus solve the scalability problem that most modern blockchains face.
Let’s explain on the Ethereum example:
The Ethereum blockchain is made up of thousands of computers, known as nodes, each of which “lends” a certain amount of hashrate to the network. The Ethereum Virtual Machine (EVM) uses this hashrate to execute smart contracts and manage decentralized applications (DApps).
Currently, Ethereum operates on a sequential execution basis, which means that each node must calculate and process each operation. As a result, the passage of a transaction through the verification process takes considerable time: Ethereum performs approximately 10 transactions per second, whereas Visa, for example, has a figure in the region of 24,000.
Because the entire registry is stored on each device, adding computers to the network does not necessarily increase efficiency, and the verification chain simply becomes longer.
The idea behind sharding is to replace the model in which each note must calculate each operation with a parallel execution model in which nodes process only certain calculations. This enables multiple transactions to be processed concurrently.
The blockchain is divided into shards (subdomains or segments). Nodes only manage the part of the registry to which they are connected (perform processes and confirm transactions), not the entire registry.
Sharding is a potential solution to the scaling problem.
The more popular blockchain becomes, the more users initiate transactions, launch decentralized applications, and engage in other network-based activities. As a result, the transaction rate falls, impeding the long-term growth of the blockchain. The increased transactional activity necessitates the node to intensify the transaction verification process. There is a risk that these blockchains will “clog up” as Ethereum did during the CryptoKitties boom when the game accounted for 11% of network transactions.
If node groups are in charge of individual segments, each node does not need to maintain the entire registry to perform each operation. As a result, transaction validation can be performed in parallel rather than in a linear manner, increasing network speed. This solves the scaling issue.
The primary issues with sharding are communication and security. If the blockchain is split into isolated segments, each shard becomes its own network. Users and applications in one subdomain will be unable to communicate with users and applications in another unless a special communication mechanism is used.
A security issue arises in the segmented blockchain because hackers can easily capture one shard due to the lower hashrate required to control individual segments. (the so-called 1% attack).
After capturing a segment, attackers can send invalid transactions to the main network. Also, the data in this particular segment may become invalid and be irretrievably lost. Ethereum offers a solution in the form of a randomized sample — shard protocols are randomly assigned to various sections to confirm the authentication of blocks.
The developers have proposed two solutions to improve the performance and speed of transactions in blockchains.
The first option is to increase the block size. The key concept is that the larger the block size, the more transactions that can be placed in it, and thus the higher the number of transactions per second.
However, the larger the block, the more computing power is required to verify it. If the block size is significantly increased, only the most powerful computers will be able to manage the computing power required to operate as a node.
Due to the high cost of such computer equipment, node pools will inevitably become smaller and more centralized, increasing the risk of an attack by 51%. Increasing the block size necessitates a hard fork, which has the potential to split the community: if not all users accept the update, two different chains using different coins will emerge. Increasing the block size may not be a long-term solution.
The second suggestion is to use altcoins so that various functions and applications can be implemented on their own networks using their own coins.
This model will increase productivity because a single blockchain will not be overloaded, but it will also increase security risks because computing power will be distributed across multiple blockchains. Again, the risk of network hacking will increase because the computing power required to carry out a 51% attack will be much lower.
Zilliqa is the first platform to use sharding. During the crowding stage, it was able to achieve an indicator of 2,828 transactions per second.
The Near blockchain ecosystem enables developers to create and deploy decentralized applications. Near bills itself as a “sharded blockchain on PoS” and claims that its sharding technology allows nodes to remain small enough to function on low-performance devices, potentially even mobile phones.
Ethereum provides a blockchain ecosystem for developing DApps based on smart contracts. The Ethereum Foundation intends to include sharding in the updated Ethereum 2.0 protocol.
Among other projects working with sharding are Cardano, QuarkChain, and PCchain.
Sharding technology appears in the white paper of the Libra digital currency. Ahead of the launch, Facebook purchased Chainspace, whose development team specializes in sharding. Specific details are still unknown, but it can be assumed that a kind of sharding will be introduced into the Libra blockchain.
Sharding can theoretically become a solution to the so-called blockchain trilemma.
The blockchain trilemma, as explained by Vitalik Buterin, is that only two of the three key features of the blockchain — security, decentralization, and scalability — can be saved at the same time. If the difficulties associated with sharding are overcome, it will be possible to scale distributed networks without sacrificing decentralization or security.
We’d love to hear your thoughts on sharding in the comments below. If you like this article, subscribe to our Medium Feed for more content like this. Stay tuned!
New to trading? Try crypto trading bots or copy trading
