how the watch-only wallet knows new addresses generated by offline wallet?

[ora.zhang]

Now I'm trying to use Electrum to store my BTC with the cold storage way.
But there are still some questions which confuse me, could someone give me hints about that ? I searched the forum and haven't found what I want.

There are two wallets and one is offline electrum wallet(W1) and the other is a online one(W2) which is generated by the master-pub key from the offline wallet. Both of them have the same addresses.
After lots of transactions are done, as I learned from electrum documents, there would be new addresses generated by W1.
The questions are:
     1. Could W2(watch-only wallet) show the correct balance info for this address ? If Yes, How does W2 know this address is generated by W1?
     2. Is it enough to restore all the balances and info from the seed without the wallet backup file after new addresses are generated? If Yes, the only one I  need care about is the wallet even after lots of transaction are done, that would be awesome.
     3. Does the change_gap_limit have bad effects of the cold storage or wallet restore?


Thanks for helping a newbie,

ora.zhang

[jackg]

In response to your question.
1. A master prblic key knows the addresses the master private key (what you don't see) is going to produce. Both of these come from the seed but the master public key has no control over the bitcoins.
2. Yes, you can restore everything from the seed.
3. No change_gap_limit should keep generating your addresses. You'll run out of addresses if you don't do that (as an unreasonable number of change addresses will be used that your wallet don't have if offline and not setting a gap limit).

[ora.zhang]

Thanks a lot for your info.

In response to your question.
1. A master prblic key knows the addresses the master private key (what you don't see) is going to produce. Both of these come from the seed but the master public key has no control over the bitcoins.

OK. But how the online wallet(W2) knows whether the new address is produced by the offline one(W1). I could use the online wallet to send transactions even the address and private key are produced by another wallet, for example, BitCore-Qt wallet. Is there any calculation or algorithm behind the checking?

2. Yes, you can restore everything from the seed.

Since the new addresses is produced after the Wallet is created, and the gap limit is 20 by default. If I recover from the seed, I think only 20 addresses are produced by default. If you are correct, is that possible the new address is not in the 'address' list by default ? I have to create new addresses and wish it shows up.

3. No change_gap_limit should keep generating your addresses. You'll run out of addresses if you don't do that (as an unreasonable number of change addresses will be used that your wallet don't have if offline and not setting a gap limit).

[HCP]

OK. But how the online wallet(W2) knows whether the new address is produced by the offline one(W1). I could use the online wallet to send transactions even the address and private key are produced by another wallet, for example, BitCore-Qt wallet. Is there any calculation or algorithm behind the checking?

No, you couldn't use an address and private key from another wallet... Electrum HD wallets DO NOT allow you to import private keys. They can only ever hold the keys/addresses generated from your seed.

So, both the online and offline wallet use the same identical Master Public Key. This is the "starting point" for address generation, so both wallets will generate identical lists of addresses... Forever!

Since the new addresses is produced after the Wallet is created, and the gap limit is 20 by default. If I recover from the seed, I think only 20 addresses are produced by default. If you are correct, is that possible the new address is not in the 'address' list by default ? I have to create new addresses and wish it shows up.

The "gap limit" is simply how many consecutive "empty" addresses need to occur before the wallet stops looking for more used addresses after the wallet is restored.

So, if you have used say 14 addresses, and then wipe your computer and restore your wallet... During the restore, it'll find the first 14 addresses have been used... Then it will find 20 empty ones and that triggers the "gap limit" functionally, so it'll stop generating and checking addresses... And you wallet will now have a total of 34 addresses in it.

Also, each time you use an address, Electrum will automatically generate a new address out to the gap limit to maintain a pool of unused addresses.

You should not need to modify the gap limit unless you're constantly generating an enormous amount of addresses and receiving payments to them "out of order"... More likely in an ecommerce setup as opposed to a cold storage scenario

[ora.zhang]

Thanks for your answers, I get the basic idea of Electrum. But I still have some questions.

OK. But how the online wallet(W2) knows whether the new address is produced by the offline one(W1). I could use the online wallet to send transactions even the address and private key are produced by another wallet, for example, BitCore-Qt wallet. Is there any calculation or algorithm behind the checking?

No, you couldn't use an address and private key from another wallet... Electrum HD wallets DO NOT allow you to import private keys. They can only ever hold the keys/addresses generated from your seed.

So, both the online and offline wallet use the same identical Master Public Key. This is the "starting point" for address generation, so both wallets will generate identical lists of addresses... Forever!

Let's create a scenario for it.  The offline wallet(W1) and watch-only wallet(W2) have the identical addresses in the beginning and they
are marked as add1, add2 .... add20 and chadd1,....chaadd5.

And I got a payment from Bob(0.4BTC) and Alice(0.6BTC) and all the payments are received by the add1. Now I send 0.3BTC to Jacky which  uses the input from Bob. After the transaction, I have 0.7BTC(0.6 in add1 and 0.1 in chaadd1).

Q1: The add1 is used, but there is still 0.6BTC in it. Is there a new add21 produced ？ I guess the answer is yes.

Q2: In the online wallet(W2), since the add1 is used, a new address is generated and the address is add21, right ?

Since the new addresses is produced after the Wallet is created, and the gap limit is 20 by default. If I recover from the seed, I think only 20 addresses are produced by default. If you are correct, is that possible the new address is not in the 'address' list by default ? I have to create new addresses and wish it shows up.

The "gap limit" is simply how many consecutive "empty" addresses need to occur before the wallet stops looking for more used addresses after the wallet is restored.

So, if you have used say 14 addresses, and then wipe your computer and restore your wallet... During the restore, it'll find the first 14 addresses have been used... Then it will find 20 empty ones and that triggers the "gap limit" functionally, so it'll stop generating and checking addresses... And you wallet will now have a total of 34 addresses in it.

Also, each time you use an address, Electrum will automatically generate a new address out to the gap limit to maintain a pool of unused addresses.

You should not need to modify the gap limit unless you're constantly generating an enormous amount of addresses and receiving payments to them "out of order"... More likely in an ecommerce setup as opposed to a cold storage scenario.

I have about 80 agencies, everyday I will receive payments. So I want to generate a least 80 address. After I read the warning in the 'http://docs.electrum.org/en/latest/faq.html#how-is-the-wallet-encrypted' page, It looks like it's a better option to change the gap_limit to 80. Any better suggestion?

[jackg]

@OP. The gap limit works linearly. What you say would work but on the first day, you'd use gaplimit=80 and on the second gaplimit=160. Other than trying to use the gaplimit variable or a for loop, note down the number you're up to.

OK. But how the online wallet(W2) knows whether the new address is produced by the offline one(W1). I could use the online wallet to send transactions even the address and private key are produced by another wallet, for example, BitCore-Qt wallet. Is there any calculation or algorithm behind the checking?

No, you couldn't use an address and private key from another wallet... Electrum HD wallets DO NOT allow you to import private keys. They can only ever hold the keys/addresses generated from your seed.

So, both the online and offline wallet use the same identical Master Public Key. This is the "starting point" for address generation, so both wallets will generate identical lists of addresses... Forever!

Since the new addresses is produced after the Wallet is created, and the gap limit is 20 by default. If I recover from the seed, I think only 20 addresses are produced by default. If you are correct, is that possible the new address is not in the 'address' list by default ? I have to create new addresses and wish it shows up.

The "gap limit" is simply how many consecutive "empty" addresses need to occur before the wallet stops looking for more used addresses after the wallet is restored.

So, if you have used say 14 addresses, and then wipe your computer and restore your wallet... During the restore, it'll find the first 14 addresses have been used... Then it will find 20 empty ones and that triggers the "gap limit" functionally, so it'll stop generating and checking addresses... And you wallet will now have a total of 34 addresses in it.

Also, each time you use an address, Electrum will automatically generate a new address out to the gap limit to maintain a pool of unused addresses.

You should not need to modify the gap limit unless you're constantly generating an enormous amount of addresses and receiving payments to them "out of order"... More likely in an ecommerce setup as opposed to a cold storage scenario

@HCP. If op wants to validate that the address in the online wallet is in an offline wallet then the gap limit must be changed in order to show the extra addresses in the new wallet.
If op received 60 deposits and then sent to an address. The change would go to  an address invisible to the first wallet and an error will occur when trying to sign offline a transaction needed. Thus, gap limit needs changing.
Otherwise you'll get something similar to an [address not found] error.

[HCP]

No, that is incorrect...

The offline wallet will still be able to sign the transaction, regardless of whether or not it has "generated" the address.

I've tested this. I've sent a bunch of transactions and generated over 10 change addresses in my "watching only" wallet.

Offline wallet only has the initial 6 change addresses.

Using coin control, on the online watching wallet, I have selected an UTXO sitting in a change address that is NOT showing in the offline wallet.

I created an unsigned transaction spending that UTXO, transferred the transaction to the offline wallet and then signed it without any error... Then transferred the signed transaction back to the online wallet and broadcast or without any problems.

The address list in offline wallet still only shows the initial 20 receive addresses and 6 change addresses.

For good measure, I then spent the resulting change from that transaction as well... And then the next one

I believe that when you create the unsigned transaction, it is including info that allows the offline wallet to derive the correct private key that it needs to sign the transaction with.

Honestly, you really shouldn't need to mess with the gap limit, unless you are restoring your wallet and you have "missing transactions" due to having generated and used extra addresses manually prior to restoring the wallet, or you want to create an address pool larger than the default of 20 receive.

And you certainly wouldn't need to update the gap limit everyday... As addresses are used and the transactions become confirmed, the wallet will generate new addresses to keep the address pool stocked.

[jackg]

Aha. Thanks HCP!
It seems that error that used to occur with offline wallets has now been resolved and probably was quite a while ago.
The private key can be calculated for an address quickly as well if that wallet has the master private key and a regular derivation path is used (like electrum's default).

[ora.zhang]

@HCP and @jackg, Thanks a lot for your clarifications and I have a better understanding about Electrum Wallet now.