A Bitcoin-native LLM: dataset, architecture and open questions

The discussion revolves around the inadequacy of relying solely on retrieval-based models for tasks that require deep domain expertise, such as those performed by protocol engineers. The primary goal is to develop a model capable of executing specific tasks related to Bitcoin protocol engineering which include script parsing and analysis, PSBT (Partially Signed Bitcoin Transaction) auditing, UTXO (Unspent Transaction Output) graph reasoning, and intent-to-script compilation. These tasks go beyond simple information retrieval; they demand execution reasoning and a deep understanding of blockchain technology, which cannot be achieved through traditional search indices or superficial data handling.

For instance, script parsing involves analyzing raw witness scripts to determine spending conditions, script class, and potential vulnerabilities like malleability vectors and sighash interactions. Similarly, PSBT auditing requires verification against its descriptor before signing to ensure accuracy and security in transaction handling. UTXO graph reasoning encompasses tracing node ancestry and identifying structural patterns within actual network data, which necessitates live analysis rather than static data interpretation. Finally, intent-to-script compilation translates natural language custody requirements into precise miniscript or descriptor formats, highlighting necessary trade-offs.

To address these specialized needs effectively, the approach should integrate both targeted training on annotated on-chain data and robust tool scaffolding around actual blockchain nodes. This strategy not only leverages the inherent labeling provided by the blockchain but also transcends the limitations posed by corpus size, focusing instead on creating a generative dataset.

A practical step forward would be to establish a benchmark to evaluate various methodologies against these advanced tasks. Proposing a small public evaluation set, comprising around 200 tasks spread across the aforementioned categories with programmatically verifiable answers, could serve as a baseline to measure the effectiveness of different modeling approaches. This benchmarking can validate whether advanced models equipped with adequate tools can bypass the need for extensive fine-tuning, or if failures in areas like script reasoning highlight where further targeted data training is essential. Such an initiative could provide clear direction on the value of investing in specific training datasets and tool enhancements for improving model performance in complex blockchain-related tasks.