SwiftSync - smarter synchronization with hints

This technique utilizes hints about which outputs remain unspent to significantly streamline the validation process, requiring less than 100MB in total for these hints. By employing a hash aggregate that confirms validation success only when it totals zero, SwiftSync efficiently negates the need for maintaining a stateful, moment-to-moment Unspent Transaction Output (UTXO) set during Initial Block Download (IBD), thus enabling order-independent operations.

The protocol operates by assigning each output a boolean hint indicating its status post-validation within the UTXO set. Outputs predicted to remain are written to disk in an append-only manner, to be used upon completion of SwiftSync. Conversely, UTXO data for outputs not expected to remain unspent are hashed and added to an aggregate. This process is mirrored for inputs, where UTXO data is hashed and removed from the aggregate, aiming for an aggregate total of zero, indicative of successful validation and accurate hints.

A challenge arises as UTXO data is available at output processing but absent during input processing. To overcome this, SwiftSync may download outputs corresponding to spent transactions for each block or leverage the assumevalid feature, allowing for hashing outpoints instead of full UTXO data. This adaptation aims at reducing bandwidth requirements and enhancing processing speed, particularly on devices with limited RAM or multiple CPU cores. Preliminary benchmarks have demonstrated a potential 5.28x speed-up, albeit in largely sequential execution phases.

Further insights into SwiftSync's mechanics, including security against the generalized birthday problem, the utilization and implications of assumevalid, inflation checks without amount data, parallel transaction order validation, BIP30 duplicate output checks without the UTXO set, and its relationship to assumeutxo, are detailed in the full write-up available at RubenSomsen's gist. This comprehensive analysis seeks to clarify operational intricacies and invites feedback to identify any overlooked areas or potential issues.