Bitcoin Forum
September 20, 2018, 03:43:57 PM *
News: ♦♦ Bitcoin Core users must update to 0.16.3 [Torrent]. More info.
 
   Home   Help Search Donate Login Register  
Pages: « 1 [2]  All
  Print  
Author Topic: Deterministic wallet compatibility matrix  (Read 13597 times)
btc4ever
Sr. Member
****
Offline Offline

Activity: 420
Merit: 250


View Profile
December 28, 2015, 09:02:30 AM
 #21

Thanks for this list.  really useful to me right now.

I'm presently trying to implement a tool that can read in an XPub and then find all the addresses that have ever been used, both receive and change.  for auditing/accounting purposes.

I quickly found that a) not all HD wallets export the xpub (eg multibit) and b) wallets are exporting the XPub from non-standard paths eg mycelium.

<rant>I don't understand why it is so hard for wallet authors to simpy follow the Bip32 spec as written.  This is people's money we are talking about.  Did we learn nothing from the browser wars?   Are they doing this specifically to waste my time investigating?  sure feels like it....</rant>

ok, so to my question:    Given only an xpub and no additional info about where it came from, is it possible to determine what location it should be at in the path?     Basically my code needs to be 100% certain that it is generating the correct addresses, and also that it can find both receive and change addresses.

<rant2>And for those wallets that do not allow exporting the xpub at all.... really?   seriously?   Exactly whose money do you think it is?  What alternative mechanism do you provide that enables 3rd party auditing/accounting?   oh really, none?   yeah, that's what I thought.</rant2>

Psst!!  Wanna make bitcoin unstoppable? Why the Only Real Way to Buy Bitcoins Is on the Streets. Avoid banks and centralized exchanges.   Buy/Sell coins locally.  Meet other bitcoiners and develop your network.   Try localbitcoins.com or find or start a buttonwood / satoshi square in your area.  Pass it on!
1537458237
Hero Member
*
Offline Offline

Posts: 1537458237

View Profile Personal Message (Offline)

Ignore
1537458237
Reply with quote  #2

1537458237
Report to moderator
1537458237
Hero Member
*
Offline Offline

Posts: 1537458237

View Profile Personal Message (Offline)

Ignore
1537458237
Reply with quote  #2

1537458237
Report to moderator
1537458237
Hero Member
*
Offline Offline

Posts: 1537458237

View Profile Personal Message (Offline)

Ignore
1537458237
Reply with quote  #2

1537458237
Report to moderator
Make a difference with your Ether.
Donate Ether for the greater good.
SPRING.WETRUST.IO
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1537458237
Hero Member
*
Offline Offline

Posts: 1537458237

View Profile Personal Message (Offline)

Ignore
1537458237
Reply with quote  #2

1537458237
Report to moderator
1537458237
Hero Member
*
Offline Offline

Posts: 1537458237

View Profile Personal Message (Offline)

Ignore
1537458237
Reply with quote  #2

1537458237
Report to moderator
btc4ever
Sr. Member
****
Offline Offline

Activity: 420
Merit: 250


View Profile
December 28, 2015, 09:51:55 AM
 #22

answering my own question, it seems the answer is yes.  The library I am using provides a getDepth() method which seems to work for my purposes.

Still, I do not see any way to access the nodes of the path, so it's unclear if the path is formatted according to bip32, 44, or something else.  

Example, from a mycelium android xpub alone I can determine that the path is like:
m/?/?/xpub

But what I want to be certain of is:
m/44/0/xpub     ( bip 44 )

- or -

m/0/0/xpub   ( bip 32, in the wrong place )

If I'm not mistaken, the only way to obtain that info is out of band.  Eg, by asking the user "What wallet software did you export this key from?"    And then I need to maintain a matrix of wallet software versions and their idiosyncracies so I can know which bip/purpose they use for exporting keys.    fun!

Hopefully someone can inform me I'm wrong about that?   Or is there some heuristic that can be used to determine whether bip32 or bip44 is used?

ok, so to my question:    Given only an xpub and no additional info about where it came from, is it possible to determine what location it should be at in the path?     Basically my code needs to be 100% certain that it is generating the correct addresses, and also that it can find both receive and change addresses.


Psst!!  Wanna make bitcoin unstoppable? Why the Only Real Way to Buy Bitcoins Is on the Streets. Avoid banks and centralized exchanges.   Buy/Sell coins locally.  Meet other bitcoiners and develop your network.   Try localbitcoins.com or find or start a buttonwood / satoshi square in your area.  Pass it on!
fbueller
Sr. Member
****
Offline Offline

Activity: 412
Merit: 250


View Profile
December 29, 2015, 09:27:11 PM
 #23

You're right; that information is lost. You can only know the parents fingerprint, and the current index. This assumes someone has the parent key, and can compare fingerprints.

You seem to share some of my issues around HD implementations. There should be a way to import your electrum xpub into GreenAddress, or absolutely *any* HD wallet cosigning applications.

It's partially a lack of a mechanism for requesting signatures and public key information (something I've drawn up before) but largely because wallet providers prefer to lock users to their platform.

Edit:

Hmm, what are you trying to achieve btw? I had this problem in an app that accepted xpubs, I just wanted to inform the users who pasted it in (as opposed to the JS option) exactly what the path was, since they likely only kept their root key and only wanted the full path..

I'd err against doing that now, and just inform them of the issue. If your app generates keys using BIP45, etc, great, you know what the path is, and probably only need to show the address index. But for the users who'd like full control of their keys, you probably shouldn't even be exposed to the full path, as you only do child derivations.

By only ever showing the public key index, you force them to maintain that information themselves, which is all BIP32 allows.

Bitwasp Developer.
btc4ever
Sr. Member
****
Offline Offline

Activity: 420
Merit: 250


View Profile
December 30, 2015, 02:52:53 AM
 #24

thx for the reply.  I'm presently writing a command-line utility that performs wallet-discovery.  The idea is to read in a master xpub and then generate all of the receive and change addresses for the wallet, checking for received tx, until the gap limit is reached.  (It also handles multiple masters as in the case of copay wallets.)

In fact, I'm using your bitwasp library, so thx for that also!

As I have read bip32, bip44, and bip45 more, I have come to realize that the fault is not entirely with wallet authors using non-standard paths.  bip32 was fine by itself, but then bip44 and 45 re-defined the path in new ways causing much confusion and no way to resolve it.  So any code that is the recipient of an xpub (which contains depth but not path or bip number) is screwed because there is no in-band mechanism to know the bip-format.   Asking users is a non-starter, except for maybe asking them which wallet software and version they are using.

Even worse, if the receiving app gets it wrong, it has no real way of knowing because new (and incorrect) keys are happily generated.  They just won't have any received funds.   No way to differentiate that from an empty wallet.

Seems really broken and bad for interop and user experience.

copay is especially crazy.  Here's a snippet for how I have to deal with them.  ( Copay 1.x )

Code:
       // copay uses a crazy constant instead of a bip45 cosigner_index.
        // why?  cuz they can.
        $copay_constant = '2147483647';
        $relpath_base = $copay_constant;
        $abspath_base = 'm/45/' . $copay_constant;
        
        $relpath = $relpath_base . "/$type/$idx";
        $abspath = $abspath_base . "/$type/$idx";


I suppose  such games might not be necessary if my app were to receive the bip39 words, but that exposes the master private key and ruins the entire idea of 3rd party audits without private keys.  And I'm not sure if that tells us the structure or not...

I am thinking we could solve this with a new bip that defines a standard format for wallet software to export master keys(s) so that necessary information is not lost, including:
  + list of xpubs  ( and/or optionally xprivs )
  + absolute path including purpose ( bip number / format identifier )
  + number of signers (in case of multisig wallets)

Basically a standard wallet interchange format.    :-)

I'm new to this hd wallet stuff.  Has anyone drafted a relevant bip already that I'm just not aware of?

Psst!!  Wanna make bitcoin unstoppable? Why the Only Real Way to Buy Bitcoins Is on the Streets. Avoid banks and centralized exchanges.   Buy/Sell coins locally.  Meet other bitcoiners and develop your network.   Try localbitcoins.com or find or start a buttonwood / satoshi square in your area.  Pass it on!
Millionero
Sr. Member
****
Offline Offline

Activity: 395
Merit: 264


View Profile
September 07, 2016, 06:34:33 PM
 #25

Following.
Necro, I know, but this thread deserves to be seen and I want to be able to find it again.
Abdussamad
Legendary
*
Offline Offline

Activity: 1862
Merit: 1064



View Profile WWW
January 22, 2017, 08:43:27 PM
 #26

breadwallet uses bip39:

https://github.com/voisine/breadwallet/issues/131#issuecomment-69028749

please correct the file. it is a very useful resource Smiley

BitcoinNewsMagazine
Legendary
*
Online Online

Activity: 1274
Merit: 1036



View Profile WWW
January 22, 2017, 10:37:38 PM
 #27

breadwallet uses bip39:

https://github.com/voisine/breadwallet/issues/131#issuecomment-69028749

please correct the file. it is a very useful resource Smiley

I think the file is correct. Breadwallet follows the bip32 default wallet layout with path m/0'/0/x, where x = each new address generated. According to the developer Breadwallet doesn't allow you to recover all possible bip39 compatible phrases, but all Breadwallet recovery phrases are bip39 compatible.

Abdussamad
Legendary
*
Offline Offline

Activity: 1862
Merit: 1064



View Profile WWW
January 24, 2017, 10:09:29 AM
 #28

According to the developer Breadwallet doesn't allow you to recover all possible bip39 compatible phrases

Where does he say that? bip39 is not the same as bip44.

BitcoinNewsMagazine
Legendary
*
Online Online

Activity: 1274
Merit: 1036



View Profile WWW
January 24, 2017, 03:58:59 PM
 #29

According to the developer Breadwallet doesn't allow you to recover all possible bip39 compatible phrases

Where does he say that? bip39 is not the same as bip44.

See https://www.reddit.com/r/Bitcoin/comments/4n0wdt/received_my_ledger_nano_today_im_extremely/d402mre/ you're welcome

Abdussamad
Legendary
*
Offline Offline

Activity: 1862
Merit: 1064



View Profile WWW
January 24, 2017, 04:50:45 PM
 #30

According to the developer Breadwallet doesn't allow you to recover all possible bip39 compatible phrases

Where does he say that? bip39 is not the same as bip44.

See https://www.reddit.com/r/Bitcoin/comments/4n0wdt/received_my_ledger_nano_today_im_extremely/d402mre/ you're welcome

No you have misunderstood. If you look at that github issue I've linked above then you can see that breadwallet supports bip32 (HD wallets) and 39 (standard for seed mnemonic/wallet words/passphrase to seed conversion) but not bip44 (key derivation standard). Wallets that use bip39 and 44 cannot be restored in breadwallet simply because the derivation path (bip44) is different from what bread wallet uses. That is what he meant by this comment on reddit.

The phrase itself (bip39) is not the problem. There is nothing in the phrase that tells you which wallet it came from so if you support one bip39 wallet you support them all.

MilkTea
Newbie
*
Offline Offline

Activity: 7
Merit: 0


View Profile
February 07, 2018, 12:48:28 AM
 #31

Where is Jaxx.io?
Spendulus
Legendary
*
Offline Offline

Activity: 2002
Merit: 1047



View Profile
February 09, 2018, 07:20:04 PM
 #32

This is a first attempt at creating a compatibility matrix for deterministic wallets. In other words, it tries to answer the questions:

When using two different wallet apps from different devs, will I have the same list of addresses and the same balance if I:
  • use the same mnemonic sentence (seed) in both?
  • export a master private key from one into the other?
  • export a master public key from one into the other (creating a watch-only wallet)?

For now, it's an Excel file available for viewing or downloading here: https://onedrive.live.com/redir?resid=584F122BA17116EE%21313.

It has four tabs. The first, "Details", lists out (hopefully) all relevant details of various wallets that might make them compatible or not with one another.

The next three are calculated from the first; they try to answer the three corresponding questions above. (Sorry, but Excel Online doesn't render vertical text correctly, so they look a bit ugly online. Either download a local file, or hover over the wallet names in the first row to read them; they're in the same order as the wallet names in the first column.)

I'm definitely interested if anyone has any input; in particular I'm not at all confident that the Details tab has everything correct, and it's probably missing some deterministic wallets that I'm unaware of. If there are any wallet devs who could take a quick look at their wallet on the first tab to see if I got anything wrong, that'd be great!

I'm also not sure that the list of requirements (spelled out on the three right-most tabs) is sufficient to guarantee compatibility.

(Also: don't rely on this without doing your own testing first!)

I'm not sure where, if anywhere, this is headed, but it'd be nice to turn this into a set of web-based tables on GitHub, perhaps something jekyll-based like this. Again, input is most welcome.

I agree it should be a GitHub project.

Very nice and innovative idea!
Pages: « 1 [2]  All
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!