bitcoin-dev

Combined summary - Compact Block Relay BIP

Combined summary - Compact Block Relay BIP

In a Bitcoin developers mailing list conversation, Rusty Russell proposed using variable-length bit encodings to reduce the size of transmitted data.

Gregory Maxwell questioned the reliability and failure rate of this approach. Russell suggested using variable-length IDs and avoiding nonces to save time. However, Maxwell disagreed, stating that it would increase the cost of collisions. The conversation also touched on the idea of using UDP instead of TCP, but Russell was not convinced. Finally, Russell mentioned adding extra bits in the sender's coding to prevent collisions.Bitcoin developers are working on a new compact block relay system that will send transactions in smaller packets. This system aims to reduce bandwidth requirements and improve transmission speed between nodes. Instead of sending all transaction details, miners would only send a unique identifier for each transaction along with a list of excluded transactions. The receiving node would then request the full transaction information if needed.Pieter explained the length of short IDs required and suggested allowing the sender to choose. He also discussed techniques for reducing the number of bytes used for txids. Matt Corallo designed a BIP for compact block relay, introducing new data structures and a variable-length integer encoding. Short transaction IDs were introduced to represent a transaction without sending a full hash. Several heuristics can be used to improve performance, such as treating short txids that match multiple mempool transactions as non-matches and verifying against the sender's mempool to check for collisions.In another email thread, Peter R raised concerns about collision attacks, and Gregory Maxwell explained the likelihood of finding two transactions with the same initial 8 bytes of txid. Peter calculated the number of possible pairs in a set of transactions and questioned how to find the pair. Later, Peter suggested a faster way to scan the set using binary search. He agreed that the attack was feasible.The conversation also involved discussions about moderation and off-topic posts, with recommendations for sending further posts to appropriate mailing lists.Overall, the Bitcoin developers are exploring various techniques to improve the network's speed and efficiency, including reducing the size of transmitted data, using variable-length encodings, and optimizing transaction identification and relay.In a discussion on the bitcoin-dev mailing list, Tom and Gregory debated over the use of service bits for indicating additional peer-to-peer feature support. Gregory argued against using service bits for negating optional parameters, while Tom believed they were the right solution. They also discussed variable length encoding and collision attacks. Matt Corallo proposed a solution called Compact Block Relay BIP to address block propagation issues, but Tom criticized it as too complicated. The discussion also touched on the use of short transaction IDs and the acknowledgment of contributors to the document.The conversation shifted to the relay protocol and its optimization for propagation time rather than bandwidth. Two other proposals, xthin block and ILBT, were mentioned as potential solutions to decrease bandwidth usage, but further information was requested. Matt Corallo introduced his Compact Block Relay BIP, which aims to reduce data transmission. He explained the protocol flow, the calculation of short transaction IDs, and backward compatibility with older clients. The implementation of the BIP can be found on Github.A user reported testing the compact block relay design for a couple of weeks and observed a significant reduction in block-bytes sent and bandwidth spikes. The measurements showed that a high percentage of blocks were transferred with low latency, even without prediction.Matt Corallo's BIP-formatted design spec for compact block relay aimed to limit on-wire bytes during block relay. The latest version of the document can be found on Github. A user who tested the design reported over 96% reduction in block-bytes sent and decreased bandwidth spikes. The user's measurements showed that a significant percentage of blocks were transferred with low latency, even without prediction.The Compact Block Relay BIP designed by Matt Corallo aims to reduce block relay time and conserve bandwidth for nodes on the P2P network. The protocol includes two modes: high-bandwidth and low-bandwidth. Several new messages and inv types are introduced, along with the use of short transaction IDs. The protocol also includes new data structures and a variable-length integer encoding. Nodes are required to follow specific rules when requesting and sending compact blocks.The document discusses the protocol for handling missing transactions in the Bitcoin network. It emphasizes the importance of validating that the header properly commits to each transaction in the block before sending a cmpctblock message. When a requested blocktxn message is received, nodes are advised to reconstruct the full block. To improve block relay time, nodes are recommended to send a sendcmpct message with the first integer set to 1 to up to three peers based on their past performance in providing blocks quickly. Additionally, all nodes should send a sendcmpct message to all appropriate peers. Nodes with limited inbound bandwidth should prioritize using MSG_CMPCT_BLOCK/getblocktxn requests when requesting blocks. When sending cmpctblock messages, nodes should limit prefilledtxn to approximately 10KB of transactions. To optimize efficiency, nodes may choose one

Discussion History

0
Matt CoralloOriginal Post
May 2, 2016 22:13 UTC
1
May 3, 2016 05:02 UTC
2
May 6, 2016 03:09 UTC
3
May 8, 2016 00:40 UTC
4
May 8, 2016 03:24 UTC
5
May 8, 2016 10:25 UTC
6
May 9, 2016 09:35 UTC
7
May 9, 2016 10:43 UTC
8
May 9, 2016 11:32 UTC
9
May 9, 2016 13:40 UTC
10
May 9, 2016 13:57 UTC
11
May 9, 2016 14:04 UTC
12
May 9, 2016 17:06 UTC
13
May 9, 2016 18:34 UTC
14
May 9, 2016 23:37 UTC
15
May 10, 2016 01:42 UTC
16
May 10, 2016 02:12 UTC
17
May 10, 2016 05:28 UTC
18
May 10, 2016 10:07 UTC
19
May 10, 2016 21:23 UTC
20
May 11, 2016 01:12 UTC
21
May 18, 2016 01:49 UTC