A Bitcoin-native LLM: dataset, architecture and open questions

The concept of developing a Bitcoin-native Language Model (LLM) that excels in "thinking in Bitcoin" is centered around enhancing the capabilities of existing general-purpose LLMs to understand and interact with Bitcoin protocol-specific elements. This includes script reasoning, UTXO graph traversal, script recommendation, and protocol-based question-and-answer tasks. Such an LLM would be adept at analyzing raw scriptPubKey or witness scripts, identifying spending patterns, explaining conditions under which the scripts can be spent, and suggesting alternative constructions based on specific custody or payment requirements.

The development of this Bitcoin-specialized LLM requires a substantial dataset aggregated from various Bitcoin-related sources. This dataset would include Bitcoin Improvement Proposals (BIPs), technical discussions from Bitcoin mailing lists and Delving threads, source annotations from Bitcoin Core, real-world annotated scripts, and miniscripts along with descriptor corpus. Moreover, generating high-quality instruction pairs that cover the core capabilities of the model is crucial. These pairs would serve as training material to enhance the model's ability to reason about the Bitcoin protocol accurately.

The proposed architecture for the Bitcoin-native LLM suggests a bifurcated approach: a fine-tuned base model trained on a static corpus to handle script analysis and offline tasks, and a tool-calling layer for live data queries which would interface with resources like Bitcoin Core RPC and mempool.space API. This structure ensures that the model can function both with and without access to live data, maintaining utility across different scenarios.

Finally, the initiative is not aimed at creating a surveillance tool for chain analysis but is intended as a resource for wallet developers, protocol researchers, and anyone involved in auditing Bitcoin transactions. The goal is to improve interpretability and provide robust developer tools without involving the model in transaction signing or handling key materials. Open questions remain regarding the sufficiency of a 7B model fine-tuned with LoRA for this task, the existence of prior labeled datasets on Bitcoin scripts, and the community’s input on establishing meaningful benchmarks for evaluating the model’s capabilities in protocol reasoning.