Feedbacks on libbitcoinkernel & bitcoin backbone

This project utilizes the core's libbitcoinkernel (version 27.0, as recalled), aiming to enhance the functionality and efficiency of Bitcoin nodes. The code, still in its demo phase, has demonstrated capability by successfully downloading blocks from a standard bitcoind peer. Impressively, it compiles correctly across both ARM (Darwin) and x86_64 (Debian 12) platforms without relying on a binding generator, thus maintaining a minimal build toolchain. The architecture of this project comprises one main process, labeled backbone, which operates the libbitcoinkernel and manages connections with network traffic through dedicated processes (block-relay and addr-relay). However, the configuration remains basic, and comprehensive testing across all networks (reg, main, test, etc.) is yet to be conducted. For those interested, the code is available for download at bitcoinbackbone.org with the head commit identified as 689e9df60.

In terms of technical insights, Antoine highlights the effectiveness of a pure C API for bridging the memory model differences between languages such as C, Rust, and CPP. A custom C API has been developed atop the libbitcoinkernel to facilitate basic block processing and chain management initialization/shutdown. Ideally, this API would maintain all state on the CPP side while enabling multi-threaded access, thereby optimizing performance and integration. Another key point discussed is the strategy for handling data structures within the block-relay process, where pre-checks using libbitcoinkernel methods could validate the syntax structure of headers before fully processing blocks. This approach allows for consistency in applying consensus rules across different processes without necessitating redundant implementations in various languages.

Furthermore, the challenge of transaction relay in a multi-process environment is addressed, particularly regarding the validation of transactions against a UTXO set that may not be uniformly accessible across processes. Antoine is considering several solutions, including techniques similar to utreexo, to ensure accurate validation without requiring proofs within the same full node boundary. This aspect underscores the complexities and potential innovations in managing state across decentralized systems.

The project reflects a significant step forward in developing more robust and efficient Bitcoin nodes, with considerations for cross-language integration, process isolation, and transaction validation. Antoine's work offers valuable insights into the ongoing evolution of Bitcoin infrastructure and the technical intricacies involved in enhancing its scalability and security.