BIP32 (Hierarchical Deterministic Wallets) code available in Java

[grau]

bitsofproof now implements BIP32 (Hierarchical Deterministic Wallets)

I plan to store the master key in BIP38 (Passphrase protected private key) format, not yet sure if extra protection is needed for the chain code. Suggestions ?

The code supports both extended private to extended private and extended public to extended public generation. If using the second option
the merchant server will not have to store private keys for receiving payments on unique addresses generated on the fly.

BIP32 code is here:
https://github.com/bitsofproof/supernode/blob/master/api/src/main/java/com/bitsofproof/supernode/api/ExtendedKey.java

[1715133848]

1715133848

[1715133848]

1715133848

[1715133848]

1715133848

[1715133848]

1715133848

[1715133848]

1715133848

[1715133848]

1715133848

[jim618]

Good stuff grau. I was hoping someone would write an implementation in Java.

Do you know if there are any 'official' test vectors for BIP32 yet ?
I notice in your junit test you do not reference any (I'm not sure they exist yet so this is no surprise).

For full interoperability of BIP32 wallets across implementations/ languages we need pretty good test coverage.

[grau]

Good stuff grau. I was hoping someone would write an implementation in Java.

Do you know if there are any 'official' test vectors for BIP32 yet ?
I notice in your junit test you do not reference any (I'm not sure they exist yet so this is no surprise).

For full interoperability of BIP32 wallets across implementations/ languages we need pretty good test coverage.

Thanks Jim, more to come

Pieter said he has some test vectors but would want to have a review by cryptographer before publishing it.

My unit tests check self consistency of the implementation that is: owner (private key) vs. read-only (only public key) wallets generate same public keys and that the key derivation hierarchy is recorded and used.

[Pieter Wuille]

I want to stress that BIP32 isn't final yet, so please don't release code that implements it.

If there is need to make changes still, I really want to avoid several different revisions in the wild.

I'll post test vectors in the BIP document as soon as I feel confident things won't change anymore.

[etotheipi]

I want to stress that BIP32 isn't final yet, so please don't release code that implements it.

If there is need to make changes still, I really want to avoid several different revisions in the wild.

I'll post test vectors in the BIP document as soon as I feel confident things won't change anymore.

What's the timeframe on that?  What is the criteria we're using to say it's final?  I ask, because I've been hitting my new wallet format pretty hard, and I realize you're right -- if there's any chance BIP32 will change, then it could cause a mess for any users that already created wallets with the old ones.

Related:  I actually ran into this in my first couple releases of Armory where I was still tweaking the wallet algorithm (this was 12+ months ago).  I created "Wallet ID" strings that are 6 bytes long, used to distinguish wallets.  The problem was, different wallet versions using the same seed were producing the same ID because it was only based on the public key of the root.  I later decided it should be based on both the root public key and the first derived key (in this case, it would be root public key, M, and M/0 public key).  This way, the ID is encoding the root and the chaining algorithm at the same time.  It seems like a small thing, but as a developer playing with different wallet versions, it made it very easy to determine whether a wallet was generated with the same chaining algorithm you are expecting.

By the way, for reference, my BIP 32 implementation in C++ using Crypto++ is here.  Rather, that's the ChildKeyDeriv function, which is the core of BIP32.  There's some test vectors there, too, but I won't make it too obvious how to find them, since we aren't promoting it yet

[iddo]

I want to stress that BIP32 isn't final yet, so please don't release code that implements it.

If there is need to make changes still, I really want to avoid several different revisions in the wild.

I'll post test vectors in the BIP document as soon as I feel confident things won't change anymore.

Please reply in the end of the Deterministic wallets thread regarding the concerns that ErebusBat and I raised.

[Deafboy]

Is BIP32 compatible with Electrum deterministic wallet?

[jim618]

Is BIP32 compatible with Electrum deterministic wallet?

No - it generates the private keys in a different way.

I would like that we reuse Electrum's way of encoding the master key and chain code into 12 words using the same vocabulary as Electrum does. This would make all the BIP32 implementations be able to understand the same seed phrase and recreate the same BIP32 wallet.

[slush]

I would like that we reuse Electrum's way of encoding the master key and chain code into 12 words using the same vocabulary as Electrum does.

We made a lot of discussion about this with Stick and we think Electrum's word list is poorly designed in many ways and we think that there's still a chance to improve it *before* BIP32+mnemonics hit the mass adoption. Stick is preparing BIP 39 which solves these problems. I'll ask him to publish the draft which he already has for ongoing discussion. As an example, similar words as woman/women, yours/yourself, yell/yellow, wall/walk, train/trail, spot/sport may became a problem for non-native speakers.

One of BIP 39 goal is to use different word list than used by Electrum, so Electrum or any other client may support both algorithms - when user write down the sentence, it is trivial to find which algorithm has been used for generating it.

I see a huge demand for clients with deterministic keys (preferably BIP32) + mnemonic between people around me. So I hope test vectors for BIP32 will be available soon...

[stick]

I would like that we reuse Electrum's way of encoding the master key and chain code into 12 words using the same vocabulary as Electrum does.

I'll ask him to publish draft which he already has for ongoing discussio

The draft is already published at https://github.com/prusnak/mnemonic but it does not yet contain new wordlists (just the electrum one). This is a work-in-progress material and the new wordlists will be assembled soon. Please read the rationale (in https://github.com/prusnak/mnemonic/blob/master/BIP_0039.txt) why we should wait for the new specification. Thanks!

[mazi]

Hi,

There are two other BIP32 implementations in Java that I know of.

One is mine, here: https://code.google.com/r/matijamazi-bitcoinj/source/browse/?name=hdw
It contains the key derivation algorithm, some tests (I got the test vectors from the Armory code, and they originate from sipa) and a high-level implementation of the wallet structure as per spec (wallets, accounts, internal and external chains). I got this working with bitcoinj in MultiBit (created a wallet with several addresses from seed, put in some funds, deleted the keys, recreated the whole wallet from seed, spent the funds). But this was several months ago and I haven't had time to upgrade this to the new versions of bitcoinj/Multibit that have been since released. (But it should be easy since the code is practically independent of bitcoinj and completely independent of MultiBit.)

Another implementation is Chris Rico's: https://code.google.com/r/chrisrico-bitcoinj-bip0032/source/detail?r=c273326f647f64295632fbbaf952d4f254ba6a1f

Since BIP 32 is not final and I've seen some change suggestions that seem to make sense (to my lay brain), eg. https://bitcointalk.org/index.php?topic=19137.msg1411989#msg1411989 , and because I've been busy, I haven't continued working on this; but I wouldn't mind a push in that direction.

[gmaxwell]

The draft is already published at https://github.com/prusnak/mnemonic but it does not yet contain new wordlists (just the electrum one).

Do you need help generating the wordlist? I wrote some crappy python code for exactly the criteria you are suggesting a few years ago:

https://people.xiph.org/~greg/wordlist.visual.py

What that does is, starting with a dictionary (I prefer to use basic english dictionaries, the one the script is currently coded to use is at this location) picks a set of words such that the first three characters are unique and such that the visual differences beyween the words in the set is maximized.

The problems with it are that it's focused on creating a pgp wordlist size dictionary, and I suspect these criteria may be harder to accomplish for an electrum size dictionary.

[stick]

https://people.xiph.org/~greg/wordlist.visual.py

IOError: [Errno 2] No such file or directory: 'ee'

The problems with it are that it's focused on creating a pgp wordlist size dictionary, and I suspect these criteria may be harder to accomplish for an electrum size dictionary.

That would be great if you tried to adapt your script and come up with a nice way of generating 1626 words that meet that criteria. I failed a couple of times because the total number is quite high :-(

[gmaxwell]

IOError: [Errno 2] No such file or directory: 'ee'

See the other link in the post.

It's quite slow, I'd do it differently now. If there is interest I can write it in C and make it fast and less dumb.

At the moment that particular version seems to only try to make the prefixes visually distinct... but that seems a bit odd in retrospect.

For that many words these constraints may work less well, with the 'ee' dictionary there it only finds 1933 distinct three character prefixes... so that constrains the optimization a lot.

[stick]

See the other link in the post.

Ah, I see.

It's quite slow, I'd do it differently now. If there is interest I can write it in C and make it fast and less dumb.

That would be great! Looking forward to your results! I really think that wise selection of the words are really crucial for mnemonics.

[Mike Hearn]

The word lists are definitely useful for writing things down and speaking them aloud. I'm not so sure they're useful for memorisation. There's a lot of psychological research on how memory works and what makes things memorable or not. Probably random unconnected words aren't much easier than raw numbers.

There's a lot of scope for exploration here - for instance if you use sentence fragments rather than random words, does it help? There's probably enough text in the gutenberg project to let you select a bunch of random sentences and string them together, giving the imagination more to play with for linking them together into one coherent memory.