Very Large HD Wallet Indexes Used - How to recover funds?

[chamfort]

Hi, we have several eCommerce stores and to separate them we used very large HD wallet indexes based on our order ID numbers. In one case the wallet index is over 1 billion. This is larger than the wallet gap limit of 20. We didn't know there was a gap limit. How can we recover these funds using the Electrum console?

[1714847922]

1714847922

[BitMaxz]

You can just increase the gap limit higher than 20.

Here's the command you need to use

Code:

wallet.change_gap_limit(50)

Replace the "50" to the number of addresses you want to show up on your addresses tab and to find the funds on the specific address. I don't know about your ID number but you must know which address in your wallet under that "ID number". So that you can use this command below if it is already in your Electrum wallet.

Code:

ismine("YOUR_ADDRESS_HERE")

If it returns true it means the address you are looking for is already shows up on addresses tab you just need to find it and recover the funds.


And I think if it is a large index of used address it will take time before it will load all of the used addresses from your wallet.

[chamfort]

Hi, since the gap is over 1.2 billion I tried to increase the gap limit to 1222333444 and it just froze my wallet. I let it run for a day and nothing changed. I don't have data on how long it takes for this command to process but if it took 0.001 seconds per wallet it would be about 14 28 days. How could we do this faster or in a more precise manner, looking for specific ranges of HD wallet indexes?

Edit: Actually it would take 28 days.
1222333444 wallets * .001 seconds per wallet * 2 = 28 days
Electrum would find our active wallets with indexes of 1.2 billion and then look forward another 1.2 billion addresses. So multiply by 2.

[HCP]

I don't think this is going to be possible and/or feasible from the Electrum console.

I think a better approach might be to see if it is possible to modify the Electrum source code and get it to start generating addresses from an index higher than 0. Possibly closer to the origin value of your order numbers. Perhaps ask on the Electrum GitHub issues page and see if the devs can assist? https://github.com/spesmilo/electrum/issues

The devs are fairly active and might be able to offer some guidance on a workaround.


In future, you'd be better off using the "account" value of the BIP32 derivation path to separate your stores, rather than such large address indexes.

[BitMaxz]

~snip~

It seems the maximum gap limit is only 15000 addresses above this Electrum will hang.

The only fastest way that I think to recover them if you know those addresses?

What I did so far as an experiment recently that you can import the public keys or addresses on Electrum with balances(As watch-only) then create a unsign transaction. Then export the unsign transaction as clipboard or copy it and import it to your Electrum wallet(Orig wallet) then sign and broadcast the transaction.

If your plan is to recover it separately you can use coinb.in to create unsigned transactions. Just make sure you know the bitcoin addresses under your wallet and then sign and broadcast the transaction with your main Electrum wallet(Orig wallet).

[nc50lc]

You can use iancoleman's BIP39 tool to generate the keys of a specific index range like 1,000,000+
but that requires you to export your Electrum's "master private key" (the same level of risk as exporting the seed phrase).

If you're interested in the procedure, here it is:

• Download https://iancoleman.io/bip39/'s source code (Offline Usage - below the page) and run it offline to an air-gap machine (never connected to the internet).
• Open/Restore your Electrum to the same offline air-gap machine, open the console and type getmasterprivate() and take note of the result.
• Copy your master private key to iancoleman's "BIP32 Root Key" text box.
• Under "Derivation path", click "BIP32" tab for legacy or "BIP141" tab then select "P2WPKH" in the Script Semantics drop-down menu for SegWit,
  Scroll down to "Derived Addresses" and confirm the first 20 addresses.
• Now to generate the keys of the specific index, type the desired index in "starting from index" textbox
  and the number of addresses/keys in the "Show" text box, then click "more rows".

That's a painstaking manual procedure but if there are just a few used addresses (like 1000's) after each store's starting index, then it's doable.
Just make sure to do the safety instruction I described above (step 1).

[pooya87]

the "gap" variable is the number of keys it creates so for example when you set it to 1000 it will create 1000 addresses and if it is 1 billion, ... which is why it freezes. if you only have 1 address at each such indexes then you should write a small script that generates them for you. all you have to do is extract your extended private key (xprv) which is also found in your wallet file and then use any bitcoin library that has a BIP32 class. any method in that  class must take an index variable, then write a loop to go through your list of indexes.

[chamfort]

You can use iancoleman's BIP39 tool to generate the keys of a specific index range like 1,000,000+
but that requires you to export your Electrum's "master private key" (the same level of risk as exporting the seed phrase).

If you're interested in the procedure, here it is:

• Download https://iancoleman.io/bip39/'s source code (Offline Usage - below the page) and run it offline to an air-gap machine (never connected to the internet).
• Open/Restore your Electrum to the same offline air-gap machine, open the console and type getmasterprivate() and take note of the result.
• Copy your master private key to iancoleman's "BIP32 Root Key" text box.
• Under "Derivation path", click "BIP32" tab for legacy or "BIP141" tab then select "P2WPKH" in the Script Semantics drop-down menu for SegWit,
  Scroll down to "Derived Addresses" and confirm the first 20 addresses.
• Now to generate the keys of the specific index, type the desired index in "starting from index" textbox
  and the number of addresses/keys in the "Show" text box, then click "more rows".

That's a painstaking manual procedure but if there are just a few used addresses (like 1000's) after each store's starting index, then it's doable.
Just make sure to do the safety instruction I described above (step 1).

This worked, I have access to the funds now. Bless you.

I generated the private keys and then sweeped them in Electrum.

[Abdussamad]

you should really be using bitcoin core for such a large number of addresses. having the blockchain locally means you can handle many more addresses

[HCP]

The problem isn't the large number of addresses... it's the fact that large numbers were used for the "address_index" value in the derivation path.

ie. something like: m/44'/0'/0'/0/1000000000

Obviously, as Electrum works sequentially from 0, getting up to the "billions" value is going to take a while (and something it wasn't really designed to do).

You can use iancoleman's BIP39 tool to generate the keys of a specific index range like 1,000,000+
...
Now to generate the keys of the specific index, type the desired index in "starting from index" textbox
and the number of addresses/keys in the "Show" text box, then click "more rows".

Hadn't noticed that Ian Coleman's tool had that "starting index" feature! brilliant stuff...