Reducing Bitcoin Full Node Storage Without Consensus Changes

The recent revisions made to BitcoinRocks from Bitcoin Core v31.1 are outlined in a detailed comparative analysis, highlighting the significant deviations and enhancements implemented in version 31.1.1 of BitcoinRocks. This analysis reveals substantial modifications encompassing both functional and performance aspects of the system.

In terms of data storage, BitcoinRocks has transitioned from using LevelDB to RocksDB, incorporating LZ4/Zstd compression techniques which allow for more efficient data handling and storage. Notably, this change also includes an updated approach to block storage, where blk*.dat records are stored either raw or Zstd-compressed, facilitating improved reindexing, pruning, block-file accounting, and index reading processes. Adjustments have been made to accurately distinguish between physical stored size and logical serialized block size, enhancing the overall database management system.

Performance improvements in BitcoinRocks are significant, with changes such as hardware-aware RocksDB/cache allocation and the implementation of parallel prevout fetching, originally proposed in Bitcoin Core PR 35295. Furthermore, the per-peer in-flight block window has been increased from 16 to 32, optimizing the amount of work available during Initial Block Download (IBD) without altering serialization, network message formats, or consensus rules. This adjustment necessitated adaptations to existing tests, such as p2p_sendheaders.py, to accommodate the new block-download policy while preserving the test's original semantics.

BitcoinRocks also introduces selectable core, conservative, and strict local transaction-policy profiles, impacting mempool/relay behaviors but not affecting block-consensus validity. These policy adjustments cater to different user needs and risk preferences without compromising the integrity of block consensus.

Additional modifications include changes to the build process, branding, and testing frameworks. These encompass naming conventions for executables, configurations, and data directories, integration of RocksDB/LZ4/Zstd into the build process, and cross-platform continuous integration (CI) enhancements. Adaptations in test fixtures were necessary to align with the new storage mechanisms and database backend, ensuring that the testing environment reflects the operational context of BitcoinRocks accurately.

Despite extensive changes in various components like validation files and node/kernel operations, there have been no alterations to core blockchain elements such as block, transaction, proof-of-work, or script-consensus validity rules. The focus remains on addressing and rectifying any arising issues through rigorous CI/testing rather than bypassing them, emphasizing a commitment to maintaining robustness and reliability in BitcoinRocks' development.