[ANNOUNCE] Electrum - Lightweight Bitcoin Client

[ThomasV]

- ecdsa.org is missing several transactions; all the other servers give a higher  balance for my wallet, and it is ecdsa that is wrong.  Is this your server?

you are right. thanks for reporting it. I am investigating it right now.

[ThomasV]

Now for another piece of information: I run electrum from the console.  I made a mistake during my initial use a couple of days ago: I cut and paste an address from the console and sent money to it, but it was one that Electrum had just a moment earlier decided to use for the "next change address" when I made a payment that moment earlier.  This meant the receipt didn't show up properly in the Received tab as change addresses don't appear there.  So I "was clever", and edited the wallet file, moving that address from the change address list to the address list, making it appear in the tab.

the software does not expect you to do this. this explains why your wallet's displayed balance is incorrect.

[Tuxavant]

...So I "was clever", and edited the wallet file...

the software does not expect you to do this. this explains why your wallet's displayed balance is incorrect.

I too cringed when I read this... And makes me want to ask... is it at all possible to "humanize" the electrum .dat files so they are more easily editable? The specific case I was wondering about is being able to manually insert an address balance when a wallet is offline so that it doesn't need to be sync'd via sneaker/flash net.

Specifically asking about a "pythonic" format with linefeeds and indentations ('n shit)

[ThomasV]

- ecdsa.org is missing several transactions; all the other servers give a higher  balance for my wallet, and it is ecdsa that is wrong.  Is this your server?

you are right. thanks for reporting it. I am investigating it right now.

it is fixed now.

I have been playing with the database yesterday (I wanted to update some statistics) an I did not notice I created a deadlock.

[ThomasV]

...So I "was clever", and edited the wallet file...

the software does not expect you to do this. this explains why your wallet's displayed balance is incorrect.

I too cringed when I read this... And makes me want to ask... is it at all possible to "humanize" the electrum .dat files so they are more easily editable? The specific case I was wondering about is being able to manually insert an address balance when a wallet is offline so that it doesn't need to be sync'd via sneaker/flash net.

Specifically asking about a "pythonic" format with linefeeds and indentations ('n shit)

we might move to a json format; it should be easier to edit. json was requested by a developer who is interested in this bounty: https://bitcointalk.org/index.php?topic=80815.0

[zebedee]

Now for another piece of information: I run electrum from the console.  I made a mistake during my initial use a couple of days ago: I cut and paste an address from the console and sent money to it, but it was one that Electrum had just a moment earlier decided to use for the "next change address" when I made a payment that moment earlier.  This meant the receipt didn't show up properly in the Received tab as change addresses don't appear there.  So I "was clever", and edited the wallet file, moving that address from the change address list to the address list, making it appear in the tab.

the software does not expect you to do this. this explains why your wallet's displayed balance is incorrect.

It's very hard for me to determine which is correct.  The balance you seem to think is incorrect is the one I thought correct.

Because change addresses are hidden; it's impossible to see which balance the two wallets disagree about.  Can you make hiding change addresses optional?

I've just noticed that one address appears twice in the change list, and that that was the cause of the issue.  I'm not sure how that happened - I didn't add anything to the change list -  but it would be nice if the client were more robust there.  

Thanks for fixing ecdsa.org; my client is not listing it so I can't confirm it matches yet.

[ThomasV]

It's very hard for me to determine which is correct.  The balance you seem to think is incorrect is the one I thought correct.

Because change addresses are hidden; it's impossible to see which balance the two wallets disagree about.  Can you make hiding change addresses optional?

I am thinking about that, but I am not sure if it's a good idea. It certainly opens a can of worms in terms of usability.
In the meantime you can view change addresses using the command line, with the -a option. try 'electrum addresses -ab'

I've just noticed that one address appears twice in the change list, and that that was the cause of the issue.  I'm not sure how that happened - I didn't add anything to the change list -  but it would be nice if the client were more robust there.  

You manually removed an address throm the list of change addresses (the one you moved to the list of normal addresses).
Electrum detected that the list of change addresses was incomplete, and it regenerated the missing address. This is why you ended up with the same address being counted twice in your balance.

I think it's a bit unfair to call this a lack of robustness. Of course I could add code to detect duplicates in order to deal with that particular issue. However it is absolutely impossible to predict all the possible ways users can mess with their wallet file; there are probably millions of ways users may corrupt their wallet if they edit it manually, and it is not possible for a developer to deal with that. If you decide to manually edit your wallet, you should understand what you're doing, and at the very least expect consequences.

Thanks for fixing ecdsa.org; my client is not listing it so I can't confirm it matches yet.

note that you don't need to wait for it to be listed, you can type it manually.

[zebedee]

It's very hard for me to determine which is correct.  The balance you seem to think is incorrect is the one I thought correct.

Because change addresses are hidden; it's impossible to see which balance the two wallets disagree about.  Can you make hiding change addresses optional?

I am thinking about that, but I am not sure if it's a good idea. It certainly opens a can of worms in terms of usability.
In the meantime you can view change addresses using the command line, with the -a option. try 'electrum addresses -ab'

I've just noticed that one address appears twice in the change list, and that that was the cause of the issue.  I'm not sure how that happened - I didn't add anything to the change list -  but it would be nice if the client were more robust there.  

You manually removed an address throm the list of change addresses (the one you moved to the list of normal addresses).
Electrum detected that the list of change addresses was incomplete, and it regenerated the missing address. This is why you ended up with the same address being counted twice in your balance.

I think it's a bit unfair to call this a lack of robustness. Of course I could add code to detect duplicates in order to deal with that particular issue. However it is absolutely impossible to predict all the possible ways users can mess with their wallet file; there are probably millions of ways users may corrupt their wallet if they edit it manually, and it is not possible for a developer to deal with that. If you decide to manually edit your wallet, you should understand what you're doing, and at the very least expect consequences.

Thanks for fixing ecdsa.org; my client is not listing it so I can't confirm it matches yet.

note that you don't need to wait for it to be listed, you can type it manually.

Ah, cool thanks for the -ab switch.  In a GUI you could always suffix the address with "(change)" or something.  Personally I find it confusing for the Receipts tab to have a total less than my balance - one naturally wonders where the rest came from.

I didn't call it a lack of robustness - I said you could be more robust  A little different.  But you should be aware that it's Electrum creating the duplicates *in the change list*, not across both lists.  It just happened again with another new address - it appeared twice in the change address list.  Yes, I realize I ultimately caused this issue by trying to be smart, but Electrum keeps compounding the error.  You understand the logic of how the client works so I'm sure you can figure out how this happens.  I understand your point about endless ways the wallet can be corrupted, and it was my fault, but double-counting (even if listed twice) seems like a basic thing to guard against.  Again - it's Electrum that has been adding the duplicates of addresses, not me.

Please take it as constructive criticism the way I intend it.  I donated 1BTC to your address 1NTiGdrGgQrA46x9dv7XKhznKgcHrhVxo a couple of days ago.  I will definitely donate more as the client improves.

I can confirm ecdsa.org is good.

Thanks again.

[ThomasV]

Ah, cool thanks for the -ab switch.  In a GUI you could always suffix the address with "(change)" or something.  Personally I find it confusing for the Receipts tab to have a total less than my balance - one naturally wonders where the rest is.

I know. this is why I was reluctant to display address balances in that list.
The problem with displaying change addresses is that it makes the concept of a Bitcoin address much more complex for a complete newbie.
Maybe we should make advanced concepts only available in console mode.

I didn't call it a lack of robustness - I said you could be more robust   A little different.  But you should be aware that it's Electrum creating the duplicates *in the change list*, not across both lists.  It just happened again with another new address - it appeared twice in the change address list.  Yes, I realize I ultimately caused this issue by trying to be smart, but Electrum keeps compounding the error.  You understand the logic of how the client works so I'm sure you can figure out how this happens.  I understand your point about endless ways the wallet can be corrupted, and it was my fault, but double-counting (even if listed twice) seems like a basic thing to guard against.  Again - it's Electrum that's now adding the duplicates of new addresses, not me.

that's quite possible, because the address generation procedure is sensitive to order. I guess you better regenerate your wallet from seed in order to fix that.

Please take it as constructive criticism the way I intend it.  I donated 1BTC to your address 1NTiGdrGgQrA46x9dv7XKhznKgcHrhVxo a couple of days ago.  I will definitely donate more as the client improves.

I can confirm ecdsa.org is good.

Thanks again.

No problem. thanks for the donation. I was just saying that if you edit your wallet manually, you should not expect the software to deal with all possible errors, because the possibilities are endless...

[zebedee]

that's quite possible, because the address generation procedure is sensitive to order. I guess you better regenerate your wallet from seed in order to fix that.

...

No problem. thanks for the donation. I was just saying that if you edit your wallet manually, you should not expect the software to deal with all possible errors, because the possibilities are endless...

Yes I've given up fighting it and done just that - regeneration - and all is cool.  It was clearly sensitive in some way to order, including I suspect the precise order of spending relative to receipts...

Point taken about all possibilities.  However I still insist that what is in some ways an accounting application shouldn't be fooled into double counting...

Cheers!

[interlagos]

I've been playing with offline mode and I have a question.
When I restore/create wallet offline the client doesn't generate the addresses and private keys until I connect to the server. Is it a design decision or just a bug?

My concern is that if I wake up two years from now with my perfectly remembered seed and for some reason Electrum infrastructure is not around (unlikely but theoretically possible), then how do I restore my private keys? I can keep the current client on a USB stick but that doesn't seem to be enough.

Also how do I generate a new address (besides 5+1 originally created)?

PS: Regarding seeds from sentences in the book, I agree it's not entirely safe but you can always salt it with your favorite pet's name and your mother's birthday Ok just kidding, never use books!

[Tuxavant]

use the -o (offline) option when working with wallets on a non-networked computer.

You can create more than 5 addresses by editing the code, but you have to remember to do that same count when/if you restore in the future. I believe the actual line/file you need to change is documented earlier in this thread.

[flatfly]

use the -o (offline) option when working with wallets on a non-networked computer.

You can create more than 5 addresses by editing the code, but you have to remember to do that same count when/if you restore in the future. I believe the actual line/file you need to change is documented earlier in this thread.

Line 257 in wallet.py:

self.gap_limit = 5           # configuration

Or, new receive addresses get automatically created by the client as you use the existing ones.

[ThomasV]

I've been playing with offline mode and I have a question.
When I restore/create wallet offline the client doesn't generate the addresses and private keys until I connect to the server. Is it a design decision or just a bug?

My concern is that if I wake up two years from now with my perfectly remembered seed and for some reason Electrum infrastructure is not around (unlikely but theoretically possible), then how do I restore my private keys? I can keep the current client on a USB stick but that doesn't seem to be enough.

without network connection, there is no way to know how many addresses need to be restored.
of course we could generate a fixed number of them, but that solution is probably worse than the problem it tries to solve...

note that if you wake up years from now and the Electrum network is no longer there, I believe the source code will still be available.
it is difficult to remove such things from the internet.

[interlagos]

without network connection, there is no way to know how many addresses need to be restored.
of course we could generate a fixed number of them, but that solution is probably worse than the problem it tries to solve...

note that if you wake up years from now and the Electrum network is no longer there, I believe the source code will still be available.
it is difficult to remove such things from the internet.

Ok I see, so it only needs to know how many addresses were used.
I would definitely prefer an option to restore the full wallet without the network.
If it's a brain-only cold-storage wallet there shouldn't be any problems remembering the amount of addresses. I could easily specify that number during restore process, and if there is more it will generate them when connected to the server. Is this possible to support?

Regarding the source code, I still think it's easier to find an old client than try to mess with a server code just to restore the wallet.

[interlagos]

...
If we assume that each book contains on average 10000 sentences (a very generous estimate), we get around 10^15 sentences to test. In contrast, a random seed with 128 bits of entropy yield 3.4x10^38 combinations. Do you understand the difference between those numbers?

So if we take two books instead of one and 4 numbers instead of 2 which is still reasonable to remember and concatenate the two sentences we will be approaching 10^15 * 10^15 = 10^30 which is more like it.
Add some magic to the process and some secret tricks and you are good to go!
Ok maybe it was easier to just remember the 12 words

[ThomasV]

...
If we assume that each book contains on average 10000 sentences (a very generous estimate), we get around 10^15 sentences to test. In contrast, a random seed with 128 bits of entropy yield 3.4x10^38 combinations. Do you understand the difference between those numbers?

So if we take two books instead of one and 4 numbers instead of 2 which is still reasonable to remember and concatenate the two sentences we will be approaching 10^15 * 10^15 = 10^30 which is more like it.
Add some magic to the process and some secret tricks and you are good to go!
Ok maybe it was easier to just remember the 12 words

bear in mind that that 10^15 estimate was an extremely generous approximation.
I guess that 99% of the books that are around us come from a much more restricted set, maybe 1 million. and they are probably much shorter than 10000 sentences

[ThomasV]

Ok I see, so it only needs to know how many addresses were used.
I would definitely prefer an option to restore the full wallet without the network.
If it's a brain-only cold-storage wallet there shouldn't be any problems remembering the amount of addresses. I could easily specify that number during restore process, and if there is more it will generate them when connected to the server. Is this possible to support?

Regarding the source code, I still think it's easier to find an old client than try to mess with a server code just to restore the wallet.

yes it would be possible to support that.

[Lumpy]

Here's my nasty hack to display change wallets in the GUI. Use at your own risk, etc. etc.

In line 646 of lib/gui_qt.py, find:

Code:

            if self.wallet.is_change(address):continue
            label = self.wallet.labels.get(address,'')

Replace it with:

Code:

            # if self.wallet.is_change(address):continue
            # label = self.wallet.labels.get(address,'')
            label = "[change]" if self.wallet.is_change(address) else self.wallet.labels.get(address,'')

I've tried it with an empty wallet and it seems to work as expected. Note that even though you can technically change the label of the [change] addresses, the code will not display it as expected.

I didn't test the various use scenarios, such as Freezing a change address, etc. All I was shooting for was for them to display in the list, nothing more.

[interlagos]

Ok I see, so it only needs to know how many addresses were used.
I would definitely prefer an option to restore the full wallet without the network.
If it's a brain-only cold-storage wallet there shouldn't be any problems remembering the amount of addresses. I could easily specify that number during restore process, and if there is more it will generate them when connected to the server. Is this possible to support?

Regarding the source code, I still think it's easier to find an old client than try to mess with a server code just to restore the wallet.

yes it would be possible to support that.

Great! Will be looking forward to it.
So basically client needs to ask how many addresses to pre-generate (5 as default seems reasonable) when creating/restoring a wallet.