BIP39 native-language display wordlists mapped to canonical English

The recent updates to the BIP39 reference repository have introduced several enhancements and clarifications aimed at improving the handling of non-English wordlists and ensuring robust recovery and interoperability options for users. A key aspect of this update is the clearer documentation of the relationship between existing non-English BIP39 wordlists and the new proposal. Unlike previous practices, the updated approach defines separate display/input lists that map deterministically back to canonical English BIP39 indexes without reusing legacy non-English BIP39 lists as display lists. This ensures that backups created with those lists remain valid and supported by wallets adopting this standard.

Detailed documentation has been added to clarify the manual recovery path. This involves a sequence from localized display words to their respective indexes, then mapping to the corresponding English BIP39 word, and finally converting it back to the canonical English BIP39 mnemonic. Such a design allows users flexibility in wallet choice since the backup can be manually converted and restored in any standard English BIP39-compatible wallet. The preservation of checksum validity is emphasized, stating that the index sequence maintains entropy and checksum integrity without altering the original BIP39 checksum calculation.

Another significant addition is the deterministic prefix statistics report for display wordlists, documented in the newly added "docs/prefix-statistics.md". This report details prefix usage across various languages, highlighting the frequency of common character prefixes and the presence of collisions. Notably, it establishes that four-character uniqueness, which holds specifically for Korean, should not be assumed generally, advising wallets to implement full-word matching to avoid ambiguity.

Normalization practices have also been refined, with the inclusion of a normalization collision check in the validator tool. It confirms zero NFKD collisions across all display wordlists, enforcing this as a critical validation criterion in continuous integration processes. However, potential issues with lossy input handling like accent stripping or case folding are noted, recommending strict input validation by wallets to prevent ambiguity.

Lastly, the draft includes recommendations for enhancing interoperability through wallet export/display behavior. It advises favoring the reversible display path over creating new legacy non-English BIP39 backups, aiming to bolster portability and uniformity in future implementations. This aligns with the broader goal of ensuring interoperable support across multiple wallet platforms, which is suggested to be formalized as a Bitcoin Improvement Proposal (BIP).

These revisions mark a substantial improvement in the documentation, precision, and usability of BIP39 standards, particularly in accommodating a multilingual user base while retaining robust security and recovery features.