[BIP Draft] Segregated Data: a prunable, script-isolated block region for data carriage

The design allows for optional pruning past a standard retention window and places the burden of storage costs on operators who choose to retain the data. This could be beneficial for off-chain data interpreters like indexers but is unsuitable for data requiring script evaluation at spend time due to its isolation from entry contents.

This initiative is based on modifications to block serialization and extends the weight formula, incorporating reference outputs which are witness v2, hold no value, are unspendable, and excluded from the UTXO set. These features enable the data to remain prunable as consensus does not require reading the entries post-validation, thus they cannot trigger any transactions. The validation of data is depth-scoped with full enforcement within the retention window and reduced checks thereafter. Several open questions remain about implementation specifics such as sharing or dedicating version and length slots for witness v2 and the benefits of committing per-reference entry lengths for transaction weight and fee rate calculations.

Operators face decisions on whether to retain or prune data after it has been processed, influencing both storage costs and the utilization of other resources. Pruning can help reduce storage needs, typically less costly compared to other resources like processing power. The implementation of new data structures requires consensus among nodes; however, this approach increases the consumption of valuable resources as every full node must process these structures.

SegData segregates certain types of data, reducing the necessity for this data during Initial Block Download (IBD) and regular storage processes. This segregation minimizes the amount of data that needs to be validated at the tip and altogether skips validation for the segregated data, potentially conserving node resources across various dimensions. The system accommodates existing practices where users prefer to keep their data on-chain despite off-chain options. Instead of using traditional methods like OP_RETURN, SegData introduces a structured, semantically deliberate region that supports consensual retention.

The inquiry into why Bitcoin node runners would store nonmonetary data without charge raises several technical and economic considerations. Storing nonmonetary data on the Bitcoin blockchain could be driven by the immutable nature of the technology, which provides a permanent record of information that cannot be altered once written. Moreover, the decentralization aspect of Bitcoin ensures that data stored on its blockchain is replicated across all nodes, contributing to robustness against data loss. However, the practicality of using the blockchain in this way is generally limited by costs and the technical design of the system, which prioritizes transactional integrity and scalability.

The recent discussions around SegData have highlighted its distinctive approach to data storage options for node operators compared to existing vectors. Unlike traditional systems, SegData provides operators with the flexibility to opt out of storing data. This feature marks a significant shift from the conventional models where such choices are not available. This flexibility could potentially lead to changes in how data storage ecosystems function, offering more control to individual node operators and possibly influencing data management strategies within networks.