MultiBit

[jim618]

Hi ErebusBat,

Yes you are correct, the lack of encryption on the MultiBit wallets is currently its biggest weakness. Some thoughts:

1) I recommend people keep their 'savings' BTC in the Satoshi client and keep relatively small smounts in MultiBit. This is what I do.   
2) Having FileVault switched on for your personal folder keeps your wallets encrypted when your machine is locked/off but does not help when you are logged in / running MultiBit.
3) Similarly you can have your wallets in a Trucrypt volume but they are still vulnerable when the Trucrypt volume is mounted.
4) You can also have MultiBit and your wallets on a USB drive, as described here:
http://multibit.org/help_runFromUSBDrive.html
Again, when your USB drive is plugged in and decrypted (if you use TruCrypt in addition) your wallets are still vulnerable to a Trojan.

Wallet encryption is my number one priority to work on as none of the above are satisfactory.

I have the UI work detailed here:
https://docs.google.com/document/d/1_8vwkQcv-Tcfx9aU-lKbWgDhAsOkpug3wiHyTo2sy0Y/edit?pli=1#heading=h.kd3tu753qxq
There are few things to do (including some necessary wallet data changes) but it should be 'just' a matter of working through them.

Jim

[1714003277]

1714003277

[1714003277]

1714003277

[1714003277]

1714003277

[1714003277]

1714003277

[1714003277]

1714003277

[ErebusBat]

Thanks for the quick reply jim.

If i knew java I would help.... but I am just a Ruby/ex-c# guy.

[Kazimir]

1) I recommend people keep their 'savings' BTC in the Satoshi client and keep relatively small smounts in MultiBit. This is what I do.   
2) Having FileVault switched on for your personal folder keeps your wallets encrypted when your machine is locked/off but does not help when you are logged in / running MultiBit.

Until MulitBit supports built-in encryption, here's a workaround: create a separate wallet with your 'savings', and (after closing MultiBit) move that to a truecrypt drive (or partition, or container, etc). Unmount this drive and don't open it again until you actually need to get money out of your savings.

Next time when you start MultiBit, you'll see a warning that your 'savings' wallet is missing. That's correct (cause this wallet is now on your truecrypt drive which is no longer mounted). Keep your pocket change in a regular wallet, and if you copied the addresses (not the private keys!) from the savings wallet when you initially created it, you can still see their balance online on various sites such as blockchain.info.

You can still keep sending money to your savings account (you only need its public address for that) without ever accessing the encrypted storage.

[ErebusBat]

1) I recommend people keep their 'savings' BTC in the Satoshi client and keep relatively small smounts in MultiBit. This is what I do.   
2) Having FileVault switched on for your personal folder keeps your wallets encrypted when your machine is locked/off but does not help when you are logged in / running MultiBit.

Until MulitBit supports built-in encryption, here's a workaround: create a separate wallet with your 'savings', and (after closing MultiBit) move that to a truecrypt drive (or partition, or container, etc). Unmount this drive and don't open it again until you actually need to get money out of your savings.

Next time when you start MultiBit, you'll see a warning that your 'savings' wallet is missing. That's correct (cause this wallet is now on your truecrypt drive which is no longer mounted). Keep your pocket change in a regular wallet, and if you copied the addresses (not the private keys!) from the savings wallet when you initially created it, you can still see their balance online on various sites such as blockchain.info.

You can still keep sending money to your savings account (you only need its public address for that) without ever accessing the encrypted storage.

Why dont you just create a new wallet on the TC drive to begin with?

[jim618]

That would also work.

[Kazimir]

Why dont you just create a new wallet on the TC drive to begin with?

Eh, right, yeah, I was under the assumption that new wallets were automatically created in the same dir as previous ones. But that's not the case, so yes, you can just create it on the TC drive right away.

[jim618]

@Kazimir
Whichever wallet you have currently highlighted when you click on 'New' it uses that wallet's directory in the File navigation dialog.

So if I have a wallet saved as, say, . . .

/users/jim/savings/xmas.wallet

. . . that is already open in MultiBit and highlighted, I can create a new one in the same directory by clicking 'New' and choosing the new name.   It will then get saved in:

/users/jim/savings


(You can change the directory in the File navigation dialog as you like too - it is just the default directory).

[ErebusBat]

Hey Jim, do you have an ETA on the encryption?

[jim618]

I think it will be in about a month.

The main things to do are:
+ move all the existing wallets over to the new format (working on that now)
+ add in the new data for the encrypted keys to the wallets (needs agreement with bitcoinj people)
+ actually write the code and do the UI changes to support it.
+ test it. test it. test it.

[jim618]

Hello All,

I have been working on the code to migrate all the existing wallets to the new protobuf format.
Before releasing it generally I would like some beta testers to try it out.
If you have a bit of time free in the next few days, please download it and test it out.

What the wallet migrate does
1) When MultiBit starts up, it sees if there are any serialised wallets (the old type).
2) If there are, it asks the user if it is OK to upgrade now.
3) If user says yes, then loop over each wallet:
3.1) Create a temporary directory and try saving the wallet as protobuf there.
3.2) Load up the test protobuf wallet that was just saved.
3.3) Compare the new and old (same balance, same keys, same transactions).
3.4) Tidy up the temporary directory once the test save is done.
4) If the test save is successful, do it for real by:
4.1) Backing up the existing wallet
4.2) Saving the wallet to protobuf, for real this time.
4.3) Loading the new, real, protobuf wallet and checking again compared to the original (same balance, keys, transactions).

If any of the wallets cannot be migrated:
+ it prints out an error message (in the new Messages window I added in v0.4.0).
+ it keeps the wallet as serialised
+ it marks the wallet with the version of MultiBit that tried to upgrade and will not try again until a new version comes out.
+ it asks the user to mail me the error message shown on screen so that I can some feedback as to where there are problems.

What I would like the beta testers to do
I would like beta testers to try it out and report back.
This is to try it out on a larger variety of wallets before I roll it out to everybody. I have tried it out with a few dozen wallets on my machine but they all tend to be quite similar so it is not comprehensive enough.

Test plan:
1) BACKUP YOUR WALLETS.
2) Download the beta from the URL given below.
3) Start up the MultiBit v0.4.1beta1 v0.4.1beta2. It should ask you at start up if you want to do the migrate.
4) Try it out.
5) The wallets should all then be converted to protobuf. If you expand the wallet detail panel on the left hand panel for each of your wallets you can see the wallet format. It should be 'protobuf'.
6) If there are any failures with the migrate, the user should still have access to their wallet (in the old format).
7) You should see identical functionality with the new wallet format. (At the moment the only apparent difference it that it is smaller and hence quicker to load).

Download URLs
The installer downloads are all on github.com (NOT on the multibit.org website).
They are are the version 0.4.1beta1 0.4.1beta2 downloads here:
https://github.com/jim618/multibit/downloads

If you do test it out, please let me know how you get on (both good news and bad news are useful feedback).

Thanks in advance !

p.s. The installers aren't fully QAed. There might be some internationalisation terms missing at the moment - just try it out in English.

[ErebusBat]

I will test it out.

On the temp dir issue: is this a sub folder of the existing wallet?  I ask because of the true crypt discussion we had earlier.

[jim618]

Hi,

No the temp dir is system specific. On a Mac it is deep in the /var directory. I delete the file after but do not secure delete it.

Hmm, if someone gets your disk they potentially would be able to undelete it. I think I will change it to be a temp subdir of where you have the wallet for reasons you outline. Then it will be in your Trucrypt container or filevault.

Will have a look for a Java secure delete too.

Good point.

Edit: I should be able to get another beta with both of these (temp directory is under where wallet is + secure delete) out tomorrow (Sunday).

[jim618]

I have just uploaded a new beta (MultiBit 0.4.1beta2) with privacy improvements to where the temporary wallets used in the migrate are stored.

It is available at github.com here:
https://github.com/jim618/multibit/downloads

The wallet migrate is basically the same as before except that:
+ the temp directory used to test the migrate is now a subdirectory of where the wallet is (and thus will be in any TruCrypt container you use).
+ a secure delete is used - the test wallet files are overwritten with random data (synchronously) before deletion.


If you could download and test out the wallet migration I would appreciate it.

edit: It also is now internationalised into German - thanks freemoney!
edit2: The messagebox asking the user to migrate was a little too narrow in German - in the released version it will look like:

[jim618]

I just had a user on MultiBit 0.4.0 who encountered the bug:

Could not load the wallet file "<wallet file>"
The error message was "java.lang.IllegalStateException Wallet contained duplicate transaction
<transaction id>"

I encountered this at the weekend (whilst working on the wallet migration) and have fixed it in the beta I have.
As it stops people accessing their BTC if it happens what I will do is:

1) QA and release the beta code with the fix in it as soon as I can, so that the fix is on the website download.
2) Temporarily switch off the wallet migration code (This is all ready to go but it can wait a few days - better to be sure that any problems are fixed before migrating people).

That is one report out of 550 downloads over a week so I imagine it is an edge condition but all the same better to get the fix out now.
The user loaded his wallet in the end by downloading the 0.4.1beta2 from https://github.com/jim618/multibit/downloads directly.

[jim618]

There is a new release of MultiBit at:

http://multibit.org


Version 0.4.1

Bug fix:
+  Fix for wallet load error "Illegal state exception - duplicate transaction"


Release info:
Scan of release checklist


Next steps:
+ I will release the code to do the wallet migration in a few days (assuming no more bugs are found with the protobuf wallets).
+ The bitcoinj mailing group have started thrashing out the details of the encrypted wallets. The discussion document (read-only) is here:
https://docs.google.com/document/d/1bWtBjzqzSubQZ926sUz5iTeMEK30aNNQXFM9z562m-s/edit#heading=h.clwmv2kkooip. PM me if you want the link to the edit/comment version and want to contribute.
+ I will start working on the encrypted wallets user interface changes in parallel to the wallet format discussions. These are specified here:
https://docs.google.com/document/d/1_8vwkQcv-Tcfx9aU-lKbWgDhAsOkpug3wiHyTo2sy0Y/edit#heading=h.kd3tu753qxq

[jim618]

I have added some more error handling code for if anything odd happens at startup.
The user now gets an 'Error' text in the bottom left of the screen and more details in the message window.

Screen shot:



(Note:The error shown is ficticious - I just "jammed a screwdriver in" to make MultiBit fail on startup).

It will appear in the next version, 0.4.2

[jim618]

I have been working on an experimental branch today of MultiBit that is quite interesting.

Currently when you do a 'Replay blockchain and transactions' and 'Import private keys' MultiBit replays the blockchain by downloading the blocks from a Satoshi client. This is quite slow and chews bandwidth.

In the experimental branch I have:
+ It caches any blocks downloaded in a subdirectory 'cache/block' (off the directory with the multibit.properties in)
+ When you do a replay or import private keys it checks if there are blocks cached: if there are it uses them.  


Pros:
+ Much quicker replay and import.

Cons:
+ You are caching the blocks so they take up space - about 100MB per 1,000 of the most recent blocks.


It won't go into the main code just yet but it is an an interesting trade off.

[jim618]

Some results from the simple cache strategy of storing the blocks as they come from the network to disk.

Size for cache (up to block 187,062) = 2.54 GB

Some stats on how long it takes to replay all the blocks from the genesis block (hooked up to half a dozen wallets):

Code:

18:01:48.319 DEBUG org.multibit.network.CacheManager - Replaying a stack of 187062 from the block cache. 
18:02:30.410 DEBUG org.multibit.network.CacheManager - The remaining block stack is of size 180000 
18:03:32.419 DEBUG org.multibit.network.CacheManager - The remaining block stack is of size 170000 
18:04:44.286 DEBUG org.multibit.network.CacheManager - The remaining block stack is of size 160000 
18:05:50.056 DEBUG org.multibit.network.CacheManager - The remaining block stack is of size 150000 
18:06:52.448 DEBUG org.multibit.network.CacheManager - The remaining block stack is of size 140000 
18:07:54.979 DEBUG org.multibit.network.CacheManager - The remaining block stack is of size 130000 
18:08:56.652 DEBUG org.multibit.network.CacheManager - The remaining block stack is of size 120000 
18:09:53.437 DEBUG org.multibit.network.CacheManager - The remaining block stack is of size 110000 
18:10:47.646 DEBUG org.multibit.network.CacheManager - The remaining block stack is of size 100000 
18:11:45.749 DEBUG org.multibit.network.CacheManager - The remaining block stack is of size 90000 
18:12:49.316 DEBUG org.multibit.network.CacheManager - The remaining block stack is of size 80000 
18:13:54.300 DEBUG org.multibit.network.CacheManager - The remaining block stack is of size 70000 
18:15:01.551 DEBUG org.multibit.network.CacheManager - The remaining block stack is of size 60000 
18:16:33.927 DEBUG org.multibit.network.CacheManager - The remaining block stack is of size 50000 
18:17:59.623 DEBUG org.multibit.network.CacheManager - The remaining block stack is of size 40000 
18:19:23.757 DEBUG org.multibit.network.CacheManager - The remaining block stack is of size 30000 
18:20:52.436 DEBUG org.multibit.network.CacheManager - The remaining block stack is of size 20000 
18:22:20.458 DEBUG org.multibit.network.CacheManager - The remaining block stack is of size 10000 
18:24:44.032 DEBUG org.multibit.network.CacheManager - The remaining block stack is of size 0 

So that is about 23 minutes for (re)processing all the transactions in the first 187062 blocks.

I am not sure that the overall data structure is the one to go for, but it certainly saves on bandwidth and speeds up replays and imports.

[ErebusBat]

Some results from the simple cache strategy of storing the blocks as they come from the network to disk.

Size for cache (up to block 187,062) = 2.54 GB

Some stats on how long it takes to replay all the blocks from the genesis block (hooked up to half a dozen wallets):

Code:

18:01:48.319 DEBUG org.multibit.network.CacheManager - Replaying a stack of 187062 from the block cache. 
18:02:30.410 DEBUG org.multibit.network.CacheManager - The remaining block stack is of size 180000 
18:03:32.419 DEBUG org.multibit.network.CacheManager - The remaining block stack is of size 170000 
18:04:44.286 DEBUG org.multibit.network.CacheManager - The remaining block stack is of size 160000 
18:05:50.056 DEBUG org.multibit.network.CacheManager - The remaining block stack is of size 150000 
18:06:52.448 DEBUG org.multibit.network.CacheManager - The remaining block stack is of size 140000 
18:07:54.979 DEBUG org.multibit.network.CacheManager - The remaining block stack is of size 130000 
18:08:56.652 DEBUG org.multibit.network.CacheManager - The remaining block stack is of size 120000 
18:09:53.437 DEBUG org.multibit.network.CacheManager - The remaining block stack is of size 110000 
18:10:47.646 DEBUG org.multibit.network.CacheManager - The remaining block stack is of size 100000 
18:11:45.749 DEBUG org.multibit.network.CacheManager - The remaining block stack is of size 90000 
18:12:49.316 DEBUG org.multibit.network.CacheManager - The remaining block stack is of size 80000 
18:13:54.300 DEBUG org.multibit.network.CacheManager - The remaining block stack is of size 70000 
18:15:01.551 DEBUG org.multibit.network.CacheManager - The remaining block stack is of size 60000 
18:16:33.927 DEBUG org.multibit.network.CacheManager - The remaining block stack is of size 50000 
18:17:59.623 DEBUG org.multibit.network.CacheManager - The remaining block stack is of size 40000 
18:19:23.757 DEBUG org.multibit.network.CacheManager - The remaining block stack is of size 30000 
18:20:52.436 DEBUG org.multibit.network.CacheManager - The remaining block stack is of size 20000 
18:22:20.458 DEBUG org.multibit.network.CacheManager - The remaining block stack is of size 10000 
18:24:44.032 DEBUG org.multibit.network.CacheManager - The remaining block stack is of size 0 

So that is about 23 minutes for (re)processing all the transactions in the first 187062 blocks.

I am not sure that the overall data structure is the one to go for, but it certainly saves on bandwidth and speeds up replays and imports.

I love it, keep up the good work Jim!

[World]

I love it, keep up the good work Jim!

+1