Breaking change in calculation of hash_serialized_2

Oct 20 - Oct 20, 2023

  • A recent email discussion raised the question of using the SHA256 hash in a snapshot file.

James suggested that instead of hashing the snapshot file separately, it would be more efficient to use the hash of the file itself as the canonical hash. This approach would eliminate any malleability issues. James argued that since the function "gettxoutsetinfo" already requires walking through the entire UTXO set to calculate the hash, it makes sense for this function to generate the actual contents of the dump file and calculate the hash of it. He provided a link to Peter Todd's website (https://petertodd.org), although the content of the linked resource was not included in the email.It is important to note that a potential malleability issue was discovered in the UTXO set dump files used by assumeutxo. The issue was caused by a bug in the serialization of UTXOs for the calculation of hash_serialized_2, which is used by Bitcoin Core to check if the UTXO set loaded from a dump file matches the expected value. A fix for this bug is currently being worked on and is scheduled for release in v26.0 in November. As a result of the fix, the serialization will change and all historical UTXO set hash results will also change after upgrading to v26.0. The version number returned in gettxoutset will be renamed to hash_serialized_3.The decision to switch the serialization completely was made due to additional potentially problematic issues found during fuzz testing. If anyone is currently using hash_serialized_2 for any security critical purposes, it is recommended to check if the bugs in the serialization code could cause issues. Switching to hash_serialized_3 or considering using MuHash is advised. For projects that rely on hash_serialized_2 and require time to upgrade and adapt to the changes, it is requested to inform the team. Although breaking changes in APIs without deprecation warning are typically avoided, it is currently believed that keeping the buggy hash_serialized_2 is not necessary as there are no known substantial use cases and its usage may even pose security risks.The team is open to reconsidering the decision if keeping hash_serialized_2 holds serious value for downstream projects. Contact can be made directly with Fabian or through commenting on the PR or the mailing list.

Bitcoin Logo

TLDR

Join Our Newsletter

We’ll email you summaries of the latest discussions from authoritative bitcoin sources, like bitcoin-dev, lightning-dev, and Delving Bitcoin.

Explore all Products

ChatBTC imageBitcoin searchBitcoin TranscriptsSaving SatoshiBitcoin Transcripts Review
Built with 🧡 by the Bitcoin Dev Project
View our public visitor count

We'd love to hear your feedback on this project?

Give Feedback