MergeSync: Parallel UTXO-set construction in assumevalid trust model

The continuous growth of the Bitcoin blockchain, now surpassing 16 years with more than 1.2 billion on-chain transactions and nearly 700GiB of data, presents significant challenges in setting up new nodes due to the extensive amount of information that must be processed. This includes a staggering number of over 3 billion Unspent Transaction Outputs (UTXOs), 95% of which are either spent or unspendable. The primary task during the initial block download (IBD) is to identify the valid UTXOs for efficient transaction verification in new blocks. The current method, which relies on sequentially downloading and validating each block, is limited by I/O capabilities rather than CPU speed, making the process time-consuming.

In an effort to address these scalability and efficiency issues, researchers have looked into various strategies, one of which is the SwiftSync approach by Ruben Somsen. However, the focus here is on a newly proposed method inspired by SwiftSync but distinct in its application. This novel approach utilizes a map/reduce strategy alongside parallel data processing to expedite the construction of the UTXO-set, significantly reducing the time required for a node to synchronize with the network under certain trust assumptions.

The algorithm begins with sharding the blockchain data across multiple cores, where each core handles an equal portion of the data to extract OutPoints from transactions, excluding provably unspendable outputs and coinbase inputs. These OutPoints are then sorted and duplicates are removed through a single pass elimination process, followed by a recursive merging of the CSV files until a singular file remains, indicating the UTXO set's cardinality at a specific block height. This process, termed MergeSync, offers a promising solution to reduce synchronization times by leveraging parallel processing capabilities and simplifying set operations.

It’s important to note, however, that this method does not involve signature processing or verification, posing a security risk unless validated by a trustless node. The proposal raises questions about potential security vulnerabilities and the need for further research to both quantify performance improvements across different hardware setups and to establish formal security assurances.