Getting my xpub balance from bitcoind

[s2]

If I have an xpub key (well ypub, but same thing) from my Trezor and I want to query this coldstorage balance using bitcoind... I can't see an obvious way to do this and feel I must be missing a trick as surely everyone does this?

In absence of bitcoind allowing an xpub key query directly, I believe we can generate a bunch of addresses from the xpub key and was planning on asking bitcoind via rpc calls the balances of these addresses however this doesn't seem straight forward.  Firstly it requires rescanning which seems like it would be unworkably slow as addresses are generated all the time.  Secondly it would indicate another bitcoin node (or at least another bitcoin wallet) would need to be created since it would conflict with my hotwallet.

I see many sites do offer exactly this capability so looking for help on how to do this.

I don't want to use any external API service like blockchain.info or trezor.io since I want to actually do this using our own tech stack.

Any help/pointers really appreciated. 

(We use node.js as a backend if anyone happens to know any node modules that may help too.)

Thanks,

[ranochigo]

Bitcoin Core offers a way for users to interact with it using their hardware wallets: https://github.com/bitcoin-core/HWI.

You need to rescan if you're querying any addresses unless you specify otherwise. Blockexplorers and other sites allows you to do so instantaneously as they parse the blockchain for the addresses and index them. Bitcoin Core does not index by addresses.

[HCP]

Have you tried creating a separate "descriptor" wallet that uses the xpub (and appropriate derivation path)? This should enable you to create a "watching-only" wallet that uses the underlying xpub+derivation path to generate and ultimately monitor addresses.

Regardless, you will need to do one rescan of the current blockchain for it to pick up all the transactions that are already associated with the addresses generated... but once the initial rescan is complete, it should just work as normal. As the addresses will be automatically generated (defaults to generating 1000, if a higher/lower limit is not specified), then you should not need to rescan again.

Also, having it in a separate wallet file means you won't affect or "pollute" your current wallet file with the watching only data.


EDIT: I have tested this today, and it works... but a couple of important notes:

- Bitcoin Core only accepts xpubs (and xprvs)... not ypubs/yprvs nor zpubs/zprvs... I used the converter here: https://jlopp.github.io/xpub-converter/ (open source)

- The xpubs provided by Trezor Suite are at the Account level (ie. m/44'/0'/0' or m/49'/0'/0' etc)

- You will need a separate wallet for each Trezor "account"... otherwise, your balances/transactions will be all mixed.

- I'm not sure if you can specify receive and change in the same descriptor... i just imported them as two descriptors... one marked external (receive) and one marked internal (change)

- for the P2SH-P2WPKH (nested segwit) Trezor accounts... the descriptor format should be:

Code: (Receive Addresses)

sh(wpkh(xpub_long_string_here/0/*))#CHECKSUM

Code: (Change Addresses)

sh(wpkh(xpub_long_string_here/1/*))#CHECKSUM

- You can calculate the CHECKSUM value for each descriptor by using getdescriptorinfo command. Note that each descriptor will have a unique checksum.

- you can import both "receive" and "change" descriptors at the same time:

Code:

importdescriptors '[{"desc":"sh(wpkh(xpub_long_string_here/0/*))#CHECKSUM","active":true,"timestamp":1514764800,"internal":false},{"desc":"sh(wpkh(xpub_long_string_here/1/*))#CHECKSUM","active":true,"timestamp":1514764800,"internal":true}]'

This will mean you only need to do one rescan.

- For the timestamps, I used https://www.epochconverter.com/ I went back in the account history, then picked a date that was at least a few days before the first transaction showing in the account history (I picked 1st Jan 2018)... and converted that to "unix epoch" format. The timestamp isn't strictly necessary... but it can reduce the rescan time by a significant amount if you have only used your wallet for a couple of years as it starts the rescan from around the timestamp, rather than at block 0 (the default value).

- I did a quick "get new receive address" test and both Trezor Suite and Bitcon Core showed the same receive address as the first "unused" address But obviously, Bitcoin Core will allow you to get multiple "new" receive addresses. So, there could be a potential "gap limit" issue if you were to generate too many receive address in Bitcoin Core and then receive coins to one of those address past the Trezor Suite gap limit (I'm not sure what this limit actually is)

[s2]

Many thanks for the reply, can I just clarify how I can do this.

Firstly I've not used wallet descriptors before so this might be a daft question but this bit here...

Code:

sh(wpkh(

I initially read that as do a shell script with a custom wpkh executable but now not sure.   Is it shell script or is this something in the wallet descriptor format I should be learning about?

[NotATether]

Code:

sh(wpkh(

I initially read that as do a shell script with a custom wpkh executable but now not sure.   Is it shell script or is this something in the wallet descriptor format I should be learning about?

Wallet descriptors are just auxiliary information to help other programs identify the script type of an address, sort of like how you can have a "1", or "3", or "bc1" address for each private key but you want to identify which address type, except each of these maps to a script type, which is what wallet descriptors try to identify.

For this wallet descriptor, the wpkh stands for P2WPKH, and the sh stands for P2SH. The wpkh is evaluated before the sh, hence it's read as P2SH-P2WPKH.

[HCP]

Many thanks for the reply, can I just clarify how I can do this.

Firstly I've not used wallet descriptors before so this might be a daft question but this bit here...

Code:

sh(wpkh(

I initially read that as do a shell script with a custom wpkh executable but now not sure.   Is it shell script or is this something in the wallet descriptor format I should be learning about?

Sorry, I forgot to link through to the descriptor documentation in my original post... You might want to have a read of the docs here: https://github.com/bitcoin/bitcoin/blob/master/doc/descriptors.md

They will do a much better job of explaining all the syntax of the various descriptors and how they work. Feel free to ask if you need any further clarification.