Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1

[zvs]

I again received:

************************
EXCEPTION: 13leveldb_error       
Database I/O error       
bitcoin in Runaway exception 

after I made a hard reset of my machine.
Harddisk is not to blame, since last time I got this error, I fsck'ed my ext4 root partition from a LiveCD and it came out clean.

(Good job I backed-up index,chainstate, and database dirs, so I restored them without having to reindex like I did last time).

i actually had to do this reindex process about half a dozen times... but i'd stop it every so often to make a backup & it finally finished.

it seemed to have some random chance to get into a loop where it gets orphan blocks over and over

[keystroke]

Starts up fine on Windows 8 and seems ok. Doing the block reindexing now. Nice work everyone!

Quick question: Why no more detach database on shutdown?

Edit: Took just under 2 hours on a i5-560m.
Client starts and stops super fast.

Is the old database removed or can I delete something to save space?

[dooglus]

Is the old database removed or can I delete something to save space?

I think the only file you can delete to save space is blkindex.dat in the same folder as wallet.dat.  That frees up around 1.8GB.

You could also delete the blk000*.dat files from the same folder, but that shouldn't free up any space, because there are links to the same files in the blocks/ subfolder.  Unless your filesystem doesn't support hardlinks, in which case deleting the blk000*.dat files in the same folder as wallet.dat will free up 5.5GB or so.

[Pieter Wuille]

Quick question: Why no more detach database on shutdown?

Because the block database(s) are LevelDB now. The BDB databases use a "database environment" (a directory shared by all databases) for journalling and consistency checks. To support moving the database files around, the files were "detached" from the environment at shutdown. Originally, this was always done. Since 0.6.1 this was made optional for the block chain databases (wallets were still always detached at shutdown), and -detachdb was introduced to re-enable it.

LevelDB databases use an entire directory by themselves, and are independent. So there is no environment to detach from anymore. The wallet in 0.8 is still BDB, and is still always detached.

Is the old database removed or can I delete something to save space?

No, it's not. But if you don't intend to go back to 0.7.x or earlier, you can delete blkindex.dat, blk0001.dat and blk0002.dat from the datadir (don't delete anything inside the blocks/ subdirectory though).

[Pieter Wuille]

i actually had to do this reindex process about half a dozen times... but i'd stop it every so often to make a backup & it finally finished.

it seemed to have some random chance to get into a loop where it gets orphan blocks over and over

If you see the reindexing finding orphans over and over, it usually means one block couldn't be found, but laters blocks can be. Since one block in their ancestry can't be connected, it is reported as an orphan and skipped. It's not an infinite loop though, it will just go over all block files. We've had some bugs in pre-releases that could cause this, but for someone first trying 0.8.0rc1, my guess is that you had a few blocks on disk that actually were corrupted.

How did you fix it? If my assumption is correct, the only solution would be waiting a few minutes for it to skip all blocks you already have, and then waiting for it to start downloading from before the invalid block.

[hazek]

Gavin & co. Can't you guys speed things up a bit? Right now we're getting a lot of good quality publicity and I think it would be a real shame to see all these new potential users get to bitcoin.org and download 7.2 which is very likely going severely disappoint them..

If there aren't any bugs, and there appear to not be any reports of bugs, can you move things a long a bit faster? Or is there something that still needs to be done?

[dooglus]

Gavin & co. Can't you guys speed things up a bit? Right now we're getting a lot of good quality publicity and I think it would be a real shame to see all these new potential users get to bitcoin.org and download 7.2 which is very likely going severely disappoint them..

If there aren't any bugs, and there appear to not be any reports of bugs, can you move things a long a bit faster? Or is there something that still needs to be done?

I'd rather have a stable release than an early release one if that's the choice.

[twobitcoins]

Am testing it now, completely fresh install on a netbook, Windows 8 64-bit, AMD E-350, 6 GB of RAM, 5400 rpm hard drive, using -dbcache=2000.

Will report back once it's done.

Bummer, midway through downloading the blockchain, with -dbcache=2000, it kept using up RAM all the way to 4GB and then crashed.

On restart, it sits at the wallet picture with "xporting blocks from block databas" (text cut off on left and right hand side).  I can see from the Task Manager that it's doing something, pegged CPU at 50%, and increasing RAM. I'll let it sit and see where it goes.

Edit: It finally came out of the splash screen and CPU usage dropped to normal, but RAM is still increasing. I've got it set to -dbcache=1000 this time, so we'll see if it sticks to that.

I closed the instance running -dbcache=1000 to see how it would handle it, and it closed no problem. I've restarted it with -dbcache=1500, and have let it run for a while. Memory use is now up to 1,660,316 KB, so, similar to my results using -dbcache=2000 where it went up to 4GB, it doesn't seem to be minding the value chosen in this setting as a limit.

I see similar issues with memory usage.  I built bitcoin-qt v0.8.0rc1 on Windows and tested it by downloading blocks from scratch, clearing the data directory each time.

On the first attempt, I ran with "dbcache=1000" and "txindex=1" in bitcoin.conf (though I'm not sure the txindex setting had any effect, because I saw "LoadBlockIndex(): transaction index disabled" in debug.log).  It eventually crashed, and there were many bad_alloc exceptions in debug.log.

On the second attempt, I did not use any special options.  Memory usage gradually increased to 2200MB while downloading blocks, and then decreased to 1650MB once downloading was finished.  Perhaps the decrease was a result of freeing the blocks stored in mapOrphanBlocks, which I believe contained several thousand blocks that had been downloaded in reverse.

It seems there is a memory leak during initial block download.  If this will affect all new users, it seems rather important to fix.

[zvs]

i actually had to do this reindex process about half a dozen times... but i'd stop it every so often to make a backup & it finally finished.

it seemed to have some random chance to get into a loop where it gets orphan blocks over and over

If you see the reindexing finding orphans over and over, it usually means one block couldn't be found, but laters blocks can be. Since one block in their ancestry can't be connected, it is reported as an orphan and skipped. It's not an infinite loop though, it will just go over all block files. We've had some bugs in pre-releases that could cause this, but for someone first trying 0.8.0rc1, my guess is that you had a few blocks on disk that actually were corrupted.

How did you fix it? If my assumption is correct, the only solution would be waiting a few minutes for it to skip all blocks you already have, and then waiting for it to start downloading from before the invalid block.

I didn't fix it per se, I just would abort the reindex process every so often and copy the files to a backup directory (hmm, starting around block 210k).  Every 2000 or 3000 blocks, I'd stop the process, copy everything over, and start it again.  Sometimes it'd get into the orphan block loop and sometimes it wouldn't...  eventually it was able to complete the whole process (w/o me changing anything in regard to what it was using to reindex)

[phatsphere]

… and just one other thing, there are no menus in linux with ubuntu's unity. does anyone test this with this (possibly most popular) window manager in linux? there is also no application icon either.

[grue]

bad_alloc = not enough ram

[dust]

I just synced the entire blockchain in about 50 minutes on a new Mac mini with an i7-3720QM (fast) and the stock 5400 rpm hd (slow) using -dbcache=4096 -par=8 and -connect to a server on the local network.  bitcoind is running in an Ubuntu VM and there have been no crashes so far.  I was also compiling and syncing a bunch of altcoins in the background.

[dooglus]

… and just one other thing, there are no menus in linux with ubuntu's unity. does anyone test this with this (possibly most popular) window manager in linux? there is also no application icon either.

I tested in Ubuntu but not using Unity.  I'll take a look now at how it works in Unity.

Edit: I don't see the menus in Unity either, but do when using XFCE4.  In Unity each application's menu bar is supposed to appear at the very top of the screen when you move the mouse up there.  Here's an example where it works, with a terminal window:



but with bitcoin 0.8rc1 the menu doesn't appear when I mouse up to the top of the screen:



In bitcoin 0.7 the menu does appear as it should:

[caveden]

Gavin & co. Can't you guys speed things up a bit? Right now we're getting a lot of good quality publicity and I think it would be a real shame to see all these new potential users get to bitcoin.org and download 7.2 which is very likely going severely disappoint them..

If there aren't any bugs, and there appear to not be any reports of bugs, can you move things a long a bit faster? Or is there something that still needs to be done?

I'd rather have a stable release than an early release one if that's the choice.

I agree, but I also agree with hazek in the sense that it's not cool to turn off new comers. So, maybe we should not direct them immediately to bitcoin-qt on bitcoin.org... maybe we should make sure they see a list of available clients before downloading any?

[novusordo]

Gavin & co. Can't you guys speed things up a bit? Right now we're getting a lot of good quality publicity and I think it would be a real shame to see all these new potential users get to bitcoin.org and download 7.2 which is very likely going severely disappoint them..

If there aren't any bugs, and there appear to not be any reports of bugs, can you move things a long a bit faster? Or is there something that still needs to be done?

I'd rather have a stable release than an early release one if that's the choice.

I agree, but I also agree with hazek in the sense that it's not cool to turn off new comers. So, maybe we should not direct them immediately to bitcoin-qt on bitcoin.org... maybe we should make sure they see a list of available clients before downloading any?

It'd be nice if the download section on the right of bitcoin.org had one link to the clients page, and nothing else. New users generally are going to click on the first download button they see, but if they are presented with multiple options they will be more inclined to either experiment a little or research the different types of clients.

[Mike Hearn]

Yeah, we want to do that too. Actually Gavin asked me to do it last night as I'm working on the website at the moment.

It needs a bit more than just a link to the Clients page, IMHO, but we can thrash out the details on a pull request.

[fornit]

I again received:

************************
EXCEPTION: 13leveldb_error       
Database I/O error       
bitcoin in Runaway exception 

after I made a hard reset of my machine.
Harddisk is not to blame, since last time I got this error, I fsck'ed my ext4 root partition from a LiveCD and it came out clean.

(Good job I backed-up index,chainstate, and database dirs, so I restored them without having to reindex like I did last time).

i actually had to do this reindex process about half a dozen times... but i'd stop it every so often to make a backup & it finally finished.

it seemed to have some random chance to get into a loop where it gets orphan blocks over and over

i get those too, and its a completely fresh windows 7 and bitcoin 0.8 install.

[mmortal03]

Am testing it now, completely fresh install on a netbook, Windows 8 64-bit, AMD E-350, 6 GB of RAM, 5400 rpm hard drive, using -dbcache=2000.

Will report back once it's done.

Bummer, midway through downloading the blockchain, with -dbcache=2000, it kept using up RAM all the way to 4GB and then crashed.

On restart, it sits at the wallet picture with "xporting blocks from block databas" (text cut off on left and right hand side).  I can see from the Task Manager that it's doing something, pegged CPU at 50%, and increasing RAM. I'll let it sit and see where it goes.

Edit: It finally came out of the splash screen and CPU usage dropped to normal, but RAM is still increasing. I've got it set to -dbcache=1000 this time, so we'll see if it sticks to that.

I closed the instance running -dbcache=1000 to see how it would handle it, and it closed no problem. I've restarted it with -dbcache=1500, and have let it run for a while. Memory use is now up to 1,660,316 KB, so, similar to my results using -dbcache=2000 where it went up to 4GB, it doesn't seem to be minding the value chosen in this setting as a limit.

I see similar issues with memory usage.  I built bitcoin-qt v0.8.0rc1 on Windows and tested it by downloading blocks from scratch, clearing the data directory each time.

On the first attempt, I ran with "dbcache=1000" and "txindex=1" in bitcoin.conf (though I'm not sure the txindex setting had any effect, because I saw "LoadBlockIndex(): transaction index disabled" in debug.log).  It eventually crashed, and there were many bad_alloc exceptions in debug.log.

On the second attempt, I did not use any special options.  Memory usage gradually increased to 2200MB while downloading blocks, and then decreased to 1650MB once downloading was finished.  Perhaps the decrease was a result of freeing the blocks stored in mapOrphanBlocks, which I believe contained several thousand blocks that had been downloaded in reverse.

It seems there is a memory leak during initial block download.  If this will affect all new users, it seems rather important to fix.

I'd run it from scratch again myself without any special options, but something about having all the connections from Bitcoin over time causes the router I share with others to slow to a halt, and I don't have access to it at the moment to reset it when this happens.

Which version of Windows are you using?