Ultraprune merged in mainline

[gmaxwell]

What will be the new dependency(s) for a typical Linux box?
Will BDB still remain a dependency also?

No new dependencies right now. BDB is still used for the wallets and will remain a dependency for the foreseeable future even after it moves off it for the wallets just for import if nothing else.

[1713498063]

1713498063

[Maged]

I've just merged my "ultraprune" branch into mainline (including Mike's LevelDB work).

Does this require downloading and re-processing the blockchain from the beginning?

Yes.  However, to save downloading, you may provide

Code:

-loadblock=DATA_DIR/blk0001.dat -loadblock=DATA_DIR/blk0002.dat

to import the old data files into the new bitcoin database backend (ultraprune/leveldb).

* "DATA_DIR" should be replaced with the directory where your blockchain was stored in <= 0.7.1.

It doesn't do this automatically? It really should before this is officially released. You could even just add them as implied bootstrap.dat files.

[Luke-Jr]

I've just merged my "ultraprune" branch into mainline (including Mike's LevelDB work).

Does this require downloading and re-processing the blockchain from the beginning?

Yes.  However, to save downloading, you may provide

Code:

-loadblock=DATA_DIR/blk0001.dat -loadblock=DATA_DIR/blk0002.dat

to import the old data files into the new bitcoin database backend (ultraprune/leveldb).

* "DATA_DIR" should be replaced with the directory where your blockchain was stored in <= 0.7.1.

It doesn't do this automatically? It really should before this is officially released. You could even just add them as implied bootstrap.dat files.

I'm sure 0.8 will do this (and more!). Seriously, we just released 0.7.1 (from master)... 0.8 is at LEAST a month off (probably longer). Git master isn't supposed to be end-user friendly and go-for-production right now. If you need something stable, use a release!

(I'm not necessarily addressing you specifically, just the general "omg it's crazy broken" sentiment I've seen lately)

[Maged]

I've just merged my "ultraprune" branch into mainline (including Mike's LevelDB work).

Does this require downloading and re-processing the blockchain from the beginning?

Yes.  However, to save downloading, you may provide

Code:

-loadblock=DATA_DIR/blk0001.dat -loadblock=DATA_DIR/blk0002.dat

to import the old data files into the new bitcoin database backend (ultraprune/leveldb).

* "DATA_DIR" should be replaced with the directory where your blockchain was stored in <= 0.7.1.

It doesn't do this automatically? It really should before this is officially released. You could even just add them as implied bootstrap.dat files.

I'm sure 0.8 will do this (and more!). Seriously, we just released 0.7.1 (from master)... 0.8 is at LEAST a month off (probably longer). Git master isn't supposed to be end-user friendly and go-for-production right now. If you need something stable, use a release!

(I'm not necessarily addressing you specifically, just the general "omg it's crazy broken" sentiment I've seen lately)

I'm not worried that it won't be added now. I was just concerned that such a simple upgrade path didn't appear to have been considered, because statements like that would have usually added a mention that those commandline options will not be needed by release in order to avoid a full re-download.

[gmaxwell]

I'm not worried that it won't be added now. I was just concerned that such a simple upgrade path didn't appear to have been considered, because statements like that would have usually added a mention that those commandline options will not be needed by release in order to avoid a full re-download.

It's been considered and discussed, including trade-offs between upgrading in a way that allows switching back to old versions (but requiring a lot of extra disk space) or cleaning up... or offering a choice at first start up.

You're reading too much into a technology and development announcement. This isn't release notes.

[Maged]

I'm not worried that it won't be added now. I was just concerned that such a simple upgrade path didn't appear to have been considered, because statements like that would have usually added a mention that those commandline options will not be needed by release in order to avoid a full re-download.

It's been considered and discussed, including trade-offs between upgrading in a way that allows switching back to old versions (but requiring a lot of extra disk space) or cleaning up... or offering a choice at first start up.

You're reading too much into a technology and development announcement. This isn't release notes.

Glad to hear it.

[Mike Hearn]

The original LevelDB branch I made did an auto-migration, actually. It was a pretty bad hack and it seems to have got lost when sipa rebased it onto ultraprune, but it's not like we don't think about these things.

[molecular]

hey guys, decided to check out ultraprune (git git://github.com/bitcoin/bitcoin from about 20 hours ago) and ran into problems:

First, I wanted to load the old block files. bitcoin-qt started up and showed me that it's downloading (or loading?) blocks. Pretty slowly (slow atom cpu here). Next morning I see this:

#> ./bitcoin-qt -loadblock=~/.bitcoin/blk0001.dat -loadblock=~/.bitcoin/blk0002.dat
Application asked to unregister timer 0x2b000010 which is not registered in this thread. Fix application.

Trying to start again gives the following popup:



And then:



And exit:

#> ./bitcoin-qt
bitcoin-qt: /usr/include/boost/thread/pthread/recursive_mutex.hpp:62: boost::recursive_mutex::~recursive_mutex(): Assertion `!pthread_mutex_destroy(&m)' failed.
Aborted

So I thought maybe I should report this here. Need more info?

[Mike Hearn]

Could you provide your debug.log file please?

The timer warning can apparently be a bug in Qt itself:

https://bugreports.qt-project.org/browse/QTBUG-18910

What version of Qt do you have installed?

[Transisto]

I'd like to have this implemented before moving anyone to a pruned chain.

https://bitcointalk.org/index.php?topic=100779.20
(bitcoind saturating upload and affecting whole connection reliability)

[gmaxwell]

I'd like to have this implemented before moving anyone to a pruned chain.

It's a good thing this has nothing to do with moving anyone to a pruned chain.

[molecular]

Could you provide your debug.log file please?

debug.log: http://pastebin.com/mDYEG2d1
db.log: http://pastebin.com/fiXu6e2L

This contains the original "crash" and 2 tries of starting it (with the popups as described above).

The timer warning can apparently be a bug in Qt itself:

https://bugreports.qt-project.org/browse/QTBUG-18910

What version of Qt do you have installed?

#> qmake --version
QMake version 2.01a
Using Qt version 4.7.2 in /usr/lib/qt4
#> pkg-config --modversion QtCore
4.7.2

[ChrisKoss]

Mods, could you fork this discussion to keep it on topic?

Atlas, please make a new thread if you wish to discuss something that is not on-topic for this thread.

[molecular]

original problem description: https://bitcointalk.org/index.php?topic=119525.msg1290009#msg1290009

Could you provide your debug.log file please?

debug.log: http://pastebin.com/mDYEG2d1
db.log: http://pastebin.com/fiXu6e2L

This contains the original "crash" and 2 tries of starting it (with the popups as described above).

The timer warning can apparently be a bug in Qt itself:

https://bugreports.qt-project.org/browse/QTBUG-18910

What version of Qt do you have installed?

#> qmake --version
QMake version 2.01a
Using Qt version 4.7.2 in /usr/lib/qt4
#> pkg-config --modversion QtCore
4.7.2

sipa suggested on #bitcoin-dev to try again with empty .bitcoin directory. I did (with loadblock= from old moved-away file)

it synced allright (30 hours), of course I had fresh empty wallet.

I then swapped in the pre-ultraprune (0.7.0 I think) wallet backup and voila: everything works.

This means I can't reproduce the error, at least not without re-loading the whole chain and the old wallet in place while doing that. Is this something I should do? Mabybe with newest git? Or just ignore the problem?

[Mike Hearn]

Well the problem is probably that your old node had got stuck on block 178441 and wasn't making any further progress. I don't know why that might have happened though, was that block a special one (p2sh or something?).

It may be there's some kind of migration bug that only occurs when your node is in a particular stuck state. I'm not sure it's worth debugging all possible permutations of this given the ease of re-initialization.

[molecular]

Well the problem is probably that your old node had got stuck on block 178441 and wasn't making any further progress. I don't know why that might have happened though, was that block a special one (p2sh or something?).

It may be there's some kind of migration bug that only occurs when your node is in a particular stuck state. I'm not sure it's worth debugging all possible permutations of this given the ease of re-initialization.

This is possible. My old node wouldn't start any more ("out of memory").

So it's probably some special case, as you suspect, and I agree it's probably not worth the effort to dig into it.

The bad thing, however: it seems (at least to me as a user) to have screwed the wallet.dat (although it says the keys can still be loaded). Some users might have forgotten to back up and run into this. I wouldn't know how to progress from this state without a backup. I'm not sure a blockchain reload with the "screwed" wallet.dat would work... maybe I could try this (fresh .bitcoin dir + "screwed" wallet.dat)

[cypherdoc]

did you devs digitally sign the file bitcoin-0.7.1-win32-setup.exe?  SHA256SUMS.asc file for 0.7.1?

nvm

[Pieter Wuille]

For those who are interested, I've created test binaries of the current development code.

Use with caution - several bugs were already fixed, and there are probably some left - so don't use for mining or merchant purposes.

The auto-import system is not yet implemented, so please run on a clean data directory. There are still issues with the inital block downloading, which require another major refactor to fix entirely, and with the recent improvements, those may now be the limiting factor. Use -connect to a trusted/local node, or use -loadblock or bootstrap.dat for initial sync, if you want to see the full potential speed.

The binaries are here. Use at your own risk.

[Diapolo]

Could you provide your debug.log file please?

debug.log: http://pastebin.com/mDYEG2d1
db.log: http://pastebin.com/fiXu6e2L

This contains the original "crash" and 2 tries of starting it (with the popups as described above).

The timer warning can apparently be a bug in Qt itself:

https://bugreports.qt-project.org/browse/QTBUG-18910

What version of Qt do you have installed?

#> qmake --version
QMake version 2.01a
Using Qt version 4.7.2 in /usr/lib/qt4
#> pkg-config --modversion QtCore
4.7.2

I'm rather sure more recent Qt version on the Linux boxes would save a few headaches...

Dia

[molecular]

Could you provide your debug.log file please?

debug.log: http://pastebin.com/mDYEG2d1
db.log: http://pastebin.com/fiXu6e2L

This contains the original "crash" and 2 tries of starting it (with the popups as described above).

The timer warning can apparently be a bug in Qt itself:

https://bugreports.qt-project.org/browse/QTBUG-18910

What version of Qt do you have installed?

#> qmake --version
QMake version 2.01a
Using Qt version 4.7.2 in /usr/lib/qt4
#> pkg-config --modversion QtCore
4.7.2

I'm rather sure more recent Qt version on the Linux boxes would save a few headaches...

Dia

no headaches here. That "timer bug" only occurs at exit time.