Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1

[Gavin Andresen]

Bitcoin version 0.8.0 release candidate 1 is now available from:
  http://sourceforge.net/projects/bitcoin/files/Bitcoin/bitcoin-0.8.0/test

This is a major release designed to improve performance and handle the
increasing volume of transactions on the network.

Please report bugs using the issue tracker at github:
  https://github.com/bitcoin/bitcoin/issues

Release-candidate 1 notes:

The OSX binary reports its version as "0.8.0rc1-1-gba1d080-beta" due to
issue https://github.com/bitcoin/bitcoin/issues/2285 . This will be fixed
before the final 0.8.0 release.

The Windows binaries could not be reproducibly built, due to issue
https://github.com/bitcoin/bitcoin/issues/2288 . This will also be fixed
before the final 0.8.0 release. The rc1 Windows binaries were built
by me (Gavin).



How to Upgrade
--------------

If you are running an older version, shut it down. Wait
until it has completely shut down (which might take a few minutes for older
versions), then run the installer (on Windows) or just copy over
/Applications/Bitcoin-Qt (on Mac) or bitcoin-qt (on Linux).

The first time you run after the upgrade a re-indexing process will be
started that will take anywhere from 30 minutes to several hours,
depending on the speed of your machine. If you have enough
memory, running with the -dbcache setting (e.g. -dbcache=1000 )
may make re-indexing faster.

Special notes for release candidate 1:
--------------------------------------

If you helped test pre-release versions, there are two changes that you
should be aware of:

1. Subdirectories in the data directory changed names; to avoid re-indexing
the blockchain, rename:
  mkdir $DATADIR/blocks && mv $DATADIR/blktree $DATADIR/blocks/index
  mv $DATADIR/coins $DATADIR/chainstate

2. The "undo file" format changed; if you see errors at startup during block
validation re-run with the -reindex flag to fix them.

Incompatible Changes
--------------------

This release no longer maintains a full index of historical transaction ids
by default, so looking up an arbitrary transaction using the getrawtransaction
RPC call will not work. If you need that functionality, you must run once
with -txindex=1 -reindex=1 to rebuild block-chain indices (see below for more
details).

Improvements
------------

Mac and Windows binaries are signed with certificates owned by the Bitcoin
Foundation, to be compatible with the new security features in OSX 10.8 and
Windows 8.

LevelDB, a fast, open-source, non-relational database from Google, is
now used to store transaction and block indices.  LevelDB works much better
on machines with slow I/O and is faster in general. Berkeley DB is now only
used for the wallet.dat file (public and private wallet keys and transactions
relevant to you).

Pieter Wuille implemented many optimizations to the way transactions are
verified, so a running, synchronized node uses much less memory and does
much less I/O. He also implemented parallel signature checking, so if you
have a multi-CPU machine all CPUs will be used to verify transactions.

New Features
------------

"Bloom filter" support in the network protocol for sending only relevant transactions to
lightweight clients.

contrib/verifysfbinaries is a shell-script to verify that the binary downloads
at sourceforge have not been tampered with. If you are able, you can help make
everybody's downloads more secure by running this occasionally to check PGP
signatures against download file checksums.

contrib/spendfrom is a python-language command-line utility that demonstrates
how to use the "raw transactions" JSON-RPC api to send coins received from particular
addresses (also known as "coin control").

New/changed settings (command-line or bitcoin.conf file)
--------------------------------------------------------

dbcache : now controls LevelDB memory usage. Running with (for example) -dbcache=1000
will use a gigabyte of memory and might make the initial blockchain download faster.

par : controls how many threads to use to validate transactions. Defaults to the number
of CPUs on your machine, use -par=1 to limit to a single CPU.

txindex : maintains an extra index of old, spent transaction ids so they will be found
by the getrawtransaction JSON-RPC method.

reindex : rebuild block and transaction indices from the downloaded block data.

New JSON-RPC API Features
-------------------------

lockunspent / listlockunspent allow locking transaction outputs for a period of time so
they will not be spent by other processes that might be accessing the same wallet.

addnode / getaddednodeinfo methods, to connect to specific peers without restarting.

importprivkey now takes an optional boolean parameter (default true) to control whether
or not to rescan the blockchain for transactions after importing a new private key.

Important Bug Fixes
-------------------

Privacy leak: the position of the "change" output in most transactions was not being
properly randomized, making network analysis of the transaction graph to identify
users' wallets easier.

Zero-confirmation transaction vulnerability: accepting zero-confirmation transactions
(transactions that have not yet been included in a block) from somebody you do not
trust is still not recommended, because there will always be ways for attackers to
double-spend zero-confirmation transactions. However, this release includes a bug
fix that makes it a little bit more difficult for attackers to double-spend a
certain type ("lockTime in the future") of zero-confirmation transaction.

Dependency Changes
------------------

Qt 4.8.3 (compiling against older versions of Qt 4 should continue to work)


Thanks to everybody who contributed to this release:
----------------------------------------------------

Alexander Kjeldaas
Andrey Alekseenko
Arnav Singh
Christian von Roques
Eric Lombrozo
Forrest Voight
Gavin Andresen
Gregory Maxwell
Jeff Garzik
Luke Dashjr
Matt Corallo
Mike Cassano
Mike Hearn
Peter Todd
Philip Kaufmann
Pieter Wuille
Richard Schwab
Robert Backhaus
Rune K. Svendsen
Sergio Demian Lerner
Wladimir J. van der Laan
burger2
default
fanquake
grimd34th
justmoon
redshark1802
tucenaber
xanatos

[Scrat Acorns]

0.8 syncs in just a few hours. It's great.

If you can test this RC please do so. The faster 0.8 is released the merrier.

Thank you for your hard work devteam.

[nexusakachus]

nice

[ciphermonk]

This is great news! 0.8 is a major milestone for the reference implementation! Thanks for all your dedicated work!

[paraipan]

Good work guys, keep it up!

Right now I would love to know how to code and contribute, but I'm just doing my part growing the bitcoin economy with small and sure steps. We're working hard here building the first Bitcoin exchange in Spain, and hope to have it up and running by the end of this month.

[phatsphere]

well, it is a fantastic improvement ... but still slow. since neither the disk is on the brink of dying, nor is the cpu above 5%, the network is the bottleneck.

i'm just curious, are there plans to speed up the data transfer rates? on weaker connections like mine, it will still take a long time to sync. this applies even more to less well connected 3rd world countries!

Is there some monitoring tool, that might also help to pinpoint problems with the networking (wireshark log?). looking into stream compression, or even other ways to compress based on references in the existing database ... and bundling up even more blocks in larger transfers is probably the only way to go?

[Mike Hearn]

Are you sure network is the bottleneck for you? Are you on an unusually slow connection? When syncing the chain at least, most nodes should be bottlenecked on disk seeks. So use iotop or something to look at that.

If network is the bottleneck then you probably want to switch from a full node to a lightweight node. The Bloom filtering that 0.8 provides can make them dramatically faster and use very little bandwidth.

[jgarzik]

My own notes:

• Pieter gets a lions share of credit, for his ultraprune work
• Please test!  Even an "it works for me" report is useful (just make sure to include your OS version and other helpful details in your "it works" post...)
• If you still have a slow block download, simply close the program and restart.  That will start the download again, with another peer.  There are still known issues with peer selection.  "restart program" is an ugly but useful workaround.

[cypherdoc]

i need those Windows binaries!

[hazek]

I'm going to sticky this for at least 14 days.

[phatsphere]

Are you sure network is the bottleneck for you? Are you on an unusually slow connection?

i used htop with the "r+w summary IO" column. there i get over a few seconds: 0, 0, ~150, ~27, 0, 0, ~150, ~20, 0, 0, ... for disk IO [edit: peak values could go up to over 1000]. I'm on a HDSPA mobile network, no port to the outside, and using a VPN w/o port forwarding. Yes, It's fine for browsing, but it has a unusual long latency.

But I'm not in a hurry, don't worry. I just did this including the VPN on purpose to test the networking speed. It's in my eyes (as an coder and admin but not a bitcoin dev.) definitely something which could be optimized. lightweight nodes are great, yes, but if you want to run your own node in a far-off country, you likely don't want it :-)

edit: watched the IO activity a bit longer. indeed, when there are somehow "harder" blocks, the IO goes up to 5000 and more -- and then still falls back to 0 for 1-2 seconds. interesting, so it might be IO, just not so obvious as before :-)

[hazek]

Btw are there any plans for adding the option to enter a username and password for a proxy?

[Stapleddiet]

My own notes:

• Please test!  Even an "it works for me" report is useful (just make sure to include your OS version and other helpful details in your "it works" post...)

Please keep repeating that, I forget that it helps you folk who are developing to know whats going on.
I only start my wallet to keep updated with the blockchain every other day, I am in Australia in an area where expensive mobile data is my only option for the net, I feel guilty when I dont seed things but hey thanks to all the folk who can.
Win7, ATI 6950 downgraded from 5 cards in my mining days. I salute you who are developing BTC.

[evoorhees]

Nice! Thank you amazing Dev Team. You fill my dreams with beautiful visions of the future, and my waking hours with endless opportunity.

I've said it before and I'll say it again, the Dev Team is at the core of the most important project occurring anywhere on the planet. Downloading now.

[oOoOo]

1. Can we now delete the legacy blk000x.dat behemoth files?

2. crashes on OSX 10.6.8 at shutdown:

Code:

Exception Type:  EXC_BAD_ACCESS (SIGBUS)
Exception Codes: KERN_PROTECTION_FAILURE at 0x0000000000000000

Thread 4 Crashed:
0   com.yourcompany.Bitcoin-Qt    	0x0023e963 CLevelDB::WriteBatch(CLevelDBBatch&, bool) + 35
1   com.yourcompany.Bitcoin-Qt    	0x0023fc86 CBlockTreeDB::WriteBlockFileInfo(int, CBlockFileInfo const&) + 102
2   com.yourcompany.Bitcoin-Qt    	0x000a1dd1 FindBlockPos(CValidationState&, CDiskBlockPos&, unsigned int, unsigned int, unsigned long long, bool) + 913
3   com.yourcompany.Bitcoin-Qt    	0x000a2474 CBlock::AcceptBlock(CValidationState&, CDiskBlockPos*) + 1092
4   com.yourcompany.Bitcoin-Qt    	0x000a39e6 ProcessBlock(CValidationState&, CNode*, CBlock*, CDiskBlockPos*) + 2758
5   com.yourcompany.Bitcoin-Qt    	0x000aa208 LoadExternalBlockFile(__sFILE*, CDiskBlockPos*) + 1384
6   com.yourcompany.Bitcoin-Qt    	0x000f4930 ThreadImport(void*) + 208
7   com.yourcompany.Bitcoin-Qt    	0x0006b9fa boost::detail::thread_data<boost::_bi::bind_t<void, void (*)(void*), boost::_bi::list1<boost::_bi::value<void*> > > >::run() + 42
8   libboost_thread-mt.dylib      	0x00b52845 boost::detail::thread_data_base::~thread_data_base() + 779
9   libSystem.B.dylib             	0x96ee8259 _pthread_start + 345
10  libSystem.B.dylib  

.

[foo]

What's the disk space requirement of LevelDB vs the old BDB? (In percent.)

[sega01]

Awesome! I'm trying to build 0.8.0rc1, but am having some problems.

Haven't looked into this a whole lot, but it looks like there's a bug with my gcc version, its usage, or how the headers are called out to. I can build up to the leveldb folder, after which it quits saying it can't find db/builder.h, which exists relative to the directory g++ is ran in. stracing g++ shows it never tries to access builder.h. I know little about C++ at all, but if I strip out the db/ in the include"", it picks it up. So it looks like g++ is treating includes in double quotes to be relative to the .cc file and not the CWD that gcc is called from.

Not sure where the bug is in exactly. Let me know if you would like any version information or help reproducing. This is with GCC 4.7.2.

Thanks,
Teran

[dansmith]

Gavin, there's a typo

1. Subdirectories in the data directory changed names; to avoid re-indexing
the blockchain, rename:
  mv $DATADIR/blktree $DATADIR/blocks/index
  mv $DATADIR/coins $DATADIR/chainstate

You actually want the content of blktree end up in blocks/index, so it should read

mv $DATADIR/blktree/* $DATADIR/blocks/index

[jgarzik]

Gavin, there's a typo

1. Subdirectories in the data directory changed names; to avoid re-indexing
the blockchain, rename:
  mv $DATADIR/blktree $DATADIR/blocks/index
  mv $DATADIR/coins $DATADIR/chainstate

You actually want the content of blktree end up in blocks/index, so it should read

mv $DATADIR/blktree/* $DATADIR/blocks/index

Strictly speaking, it is not a typo; $DATADIR/blocks simply has to exist first, in order for it to work.

So the instructions need "mkdir $DATADIR/blocks"

[Pieter Wuille]

i'm just curious, are there plans to speed up the data transfer rates? on weaker connections like mine, it will still take a long time to sync. this applies even more to less well connected 3rd world countries!

There are known problems with the current  block synchronisation mechanism. Not exactly a bug, but just a crappy implementation. It's certainly intended to be changed, but it's not a trivial thing to do. Using the bootstrap.dat torrent, or using -connect=<ip> to a known fast node (not necessarily a trusted one) can speed up the initial download a lot. Not really nice solutions, but for now, it's all we have.

looking into stream compression, or even other ways to compress based on references in the existing database ... and bundling up even more blocks in larger transfers is probably the only way to go?

Block compression is certainly possible, but it's hard to get significant improvements. With a lot of effort and a very specifically designed compressed format, I expect we may get to something like 40%. Unfortunately, the blocks contain lots of hashes and cryptographic signatures, which are essentially uncompressible. Right now, I don't think these are a priority. Fixing the download mechanism will have a much larger impact.

1. Can we now delete the legacy blk000x.dat behemoth files?

Yes, you can if you don't intend to downgrade to 0.7.x (or below) code anymore. Note that the old and the new block files are hardlinked on supported platforms/filesystems, so you won't actually gain any space by deleting them. You can however delete blkindex.dat.

What's the disk space requirement of LevelDB vs the old BDB? (In percent.)

There's no simple answer to this, as it's not just a change from BDB to LevelDB: the actual layout and organisation of the databases changed as well. The actual blocks haven't changed (they are in different, more and smaller files, but the data is the same). The old blkindex.dat (BDB, ~1650 MiB) was replaced by:

• the block index (LevelDB, $DATADIR/blocks/index, ~ 30 MiB)
• the chain state (LevelDB, $DATADIR/chainstate, ~ 170 MiB)
• the undo data (custom format, $DATADIR/blocks/rev*.dat, ~ 700 MiB)

So, roughly, the old index data now takes half as much space, but it's not entirely a fair comparison (we actually do store less information now).