Gossip Observer: New project to monitor the Lightning P2P network

The discussion revolves around the potential transition of the Lightning Network (LN) peer-to-peer (P2P) network from a message flooding protocol to a more efficient design, drawing inspiration from the Erlay paper and Bitcoin Improvement Proposal (BIP). This approach would likely benefit the LN by reducing latency and propagation delay, which are not as critical in LN gossip due to the tolerance for slightly outdated fee information. The nature of gossip messages being publicly signed eliminates the need for sender anonymity within the network's topology.

A focal point of this transition is the adaptation of Minisketch for LN gossip, emphasizing the need for a mapping mechanism that is resistant to collisions while keeping the data elements short to minimize reconciliation time. The Erlay paper suggests generating unique salts for each peer connection combined with transaction identifiers (TXIDs) to create distinct Minisketch set elements through a fast hash function like SipHash. This method scales hashing operations with the number of peer connections and reduces computational requirements by advocating for shorter set elements. An alternative involving the channel's short ID and block height was considered but found potentially problematic due to low entropy in certain components, highlighting the superiority of the Erlay-inspired approach.

Moreover, the proposed gossip 1.5 format introduces the possibility of including optional fields in messages, allowing nodes to tailor the information sent based on mutually supported features. This evolution aims at eliminating the need for message flooding entirely in favor of set reconciliation methods to manage gossip propagation, simplifying implementation and reducing bandwidth consumption. However, challenges such as disparate node filtering policies could lead to inefficiencies and failed reconciliations, suggesting a need for standardized practices.

Looking forward, engaging with LN implementers on the feasibility of these changes is crucial. Concerns include the management of per-peer state, the practicality of frequent set reconciliations, and unaddressed issues in current designs. Feedback will guide the development of a concrete proposal or draft BIP, marking a significant step towards enhancing LN's efficiency and scalability.