Hornet UTXO(1): A custom, constant-time, highly parallel UTXO database

The discussion revolves around the technical specifics of handling Unspent Transaction Outputs (UTXOs) within a blockchain system, focusing on storage and performance optimization strategies. The current UTXO index size is 48 bytes per entry, which is considered optimal for alignment purposes despite the possibility of reducing the data structure to 44 bytes. Given the approximate count of 175 million UTXOs, the total storage requirement amounts to 7.8 GiB, a capacity manageable by current hardware specifications. However, there's an acknowledgment of potential scalability issues; should the UTXO count significantly increase, the solution proposed involves transitioning the index storage from RAM to disk. This approach would necessitate an I/O page read per query, with the aim to mitigate additional latency through parallel processing during the Initial Block Download (IBD) phase, thereby reducing the dependency on substantial RAM for performance.

Further details include the current on-disk Transaction Output (TXO) database size, which stands at 134 GiB. Notably, this figure encompasses unspendable outputs like OP_RETURN, which have yet to be excluded through coding improvements. There's also a mention of the lack of effort towards compressing this data, although it’s unclear if such efforts would yield significant benefits. A future strategy under consideration is to streamline the storage post-IBD by generating a new table exclusive to unspent outputs, effectively condensing the data. This process could either be automatic or manually triggered by users aiming to optimize storage efficiency, particularly relevant for systems with limited resources.

The correspondence reflects on whether workstation or server-class hardware would benefit most from these optimizations, suggesting an assumption that such equipment might be better suited to handle the parallel processing requirements of this approach. This perspective invites further discussion on the balance between hardware capabilities and software optimization in managing blockchain data efficiently.