Satoshi Client: Improving time from "Start" to "Usable"

[AlphaWolf]

I run Bitcoin-qt on multiple computers, and sometimes one machine will go a few days before being turned on.  One of the frustrating things about using Bitcoin-qt is that, after waiting what feels like 5-10 minutes for the program to completely verify the block chain, you may have to wait another 20 minutes for the rest of the blockchain to download and verify.  I believe some time could be saved, but one of the devs will have to confirm.

Probably on every machine, the bottleneck during startup is the hard disk.  The CPU is relatively idle, and the network isn't being used at all.  I would guess that 99.9% of the time, it would be a safe assumption that the stored blockchain and database have not been corrupted.  Would it be possible to look at the last block stored locally, and begin "prefetching" the rest of the blockchain from any nodes the client can see?  The local blockchain would of course still be verified as usual, but you'd save a few minutes by having the new data already downloaded by the time the rest of the chain is verified.  The new data would just be cached in RAM (or disk if there was enough of it) until the verification process caught up.

Thoughts?  Possible?

[1713452674]

1713452674

[1713452674]

1713452674

[Mike Hearn]

Well, it may also be that you connected to a slow peer that can't shovel you the bits fast enough.

But what you could do is just run Bitcoin-Qt on one computer and use MultiBit on the rest. If you force MultiBit to connect to your one bqt computer, the security level is the same.

[AlphaWolf]

Well, it may also be that you connected to a slow peer that can't shovel you the bits fast enough.

But what you could do is just run Bitcoin-Qt on one computer and use MultiBit on the rest. If you force MultiBit to connect to your one bqt computer, the security level is the same.

I'll take a look at MultiBit.  I do have one "always on" full-participatory node, and the desktops already sync to that machine... but laptops being mobile,  they sync with the network at large.  Thanks for the tip

Of course, that doesn't really answer my question .  I believe Bitcoin-Qt would be more user friendly if it took advantage of idle network time during database verification, and fetched all the new blocks.  Could that be done?  (In other words... "This is a feature request/suggestion")  Just some quick math...

Average block size is currently: 159,450 (I assume bytes?)
Blocks per 24 hr: 144
Bytes per day: 22,960,800
Low-end broadband (6Mbit) = 786,432 Bps
22,960,800 / 786,432 = 29 seconds for each day a client is behind.

Since it can take a few minutes for the database to verify, even if your client is a few days behind, it would be ready as soon as the database was verified.  Saving users several seconds to potentially minutes seems like a marked and worthwhile improvement to me.  It's just a suggestion, if it isn't terribly difficult to to implement. 

[TierNolan]

What does database verification involve?

[grue]

What does database verification involve?

verification of all transactions in the block. this usually isn't a limiting factor because of multithreaded verification introduced in v0.8.0. The number limiting factor is usually blockchain download (from slow peer, since download isn't parallel). If you allocate enough ram for the db cache (2 gb+), disk usage shouldn't be a limiting factor.

[TierNolan]

verification of all transactions in the block. this usually isn't a limiting factor because of multithreaded verification introduced in v0.8.0.

The number limiting factor is usually blockchain download (from slow peer, since download isn't parallel). If you allocate enough ram for the db cache (2 gb+), disk usage shouldn't be a limiting factor.

However, it looks like the client does something when you start it up. 

Does it scan all the entire already verified database?

Once it starts, it then begins downloading.

[AlphaWolf]

What does database verification involve?

verification of all transactions in the block. this usually isn't a limiting factor because of multithreaded verification introduced in v0.8.0. The number limiting factor is usually blockchain download (from slow peer, since download isn't parallel). If you allocate enough ram for the db cache (2 gb+), disk usage shouldn't be a limiting factor.

At startup its certainly is...  which is what this is about.  Slow peer or not, if the fresh blocks were being downloaded while the database was being verified, a user wouldn't have to wait on *both*.  Even with a slow peer, the fresh data would likely be downloaded by the time the local database was verified.  That means once the database is verified, the user is welcomed to a fully updated and functional GUI, instead of a progress bar and "(out of sync)".

[Mike Hearn]

Yeah it could be optimised, but it'd go from "very slow" to "slightly less slow". At some point optimising Bitcoin-Qt initial bootstrap becomes uninteresting. Unless you believe there are a class of people who would wait a day to bring up a new node but not 1.5 days?