Bitcoin Forum

Bitcoin => Bitcoin Discussion => Topic started by: Gavin Andresen on February 09, 2013, 03:52:08 PM



Title: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: Gavin Andresen on February 09, 2013, 03:52:08 PM
Bitcoin version 0.8.0 release candidate 1 is now available from:
  http://sourceforge.net/projects/bitcoin/files/Bitcoin/bitcoin-0.8.0/test

This is a major release designed to improve performance and handle the
increasing volume of transactions on the network.

Please report bugs using the issue tracker at github:
  https://github.com/bitcoin/bitcoin/issues

Release-candidate 1 notes:

The OSX binary reports its version as "0.8.0rc1-1-gba1d080-beta" due to
issue https://github.com/bitcoin/bitcoin/issues/2285 . This will be fixed
before the final 0.8.0 release.

The Windows binaries could not be reproducibly built, due to issue
https://github.com/bitcoin/bitcoin/issues/2288 . This will also be fixed
before the final 0.8.0 release. The rc1 Windows binaries were built
by me (Gavin).



How to Upgrade
--------------

If you are running an older version, shut it down. Wait
until it has completely shut down (which might take a few minutes for older
versions), then run the installer (on Windows) or just copy over
/Applications/Bitcoin-Qt (on Mac) or bitcoin-qt (on Linux).

The first time you run after the upgrade a re-indexing process will be
started that will take anywhere from 30 minutes to several hours,
depending on the speed of your machine. If you have enough
memory, running with the -dbcache setting (e.g. -dbcache=1000 )
may make re-indexing faster.

Special notes for release candidate 1:
--------------------------------------

If you helped test pre-release versions, there are two changes that you
should be aware of:

1. Subdirectories in the data directory changed names; to avoid re-indexing
the blockchain, rename:
  mkdir $DATADIR/blocks && mv $DATADIR/blktree $DATADIR/blocks/index
  mv $DATADIR/coins $DATADIR/chainstate

2. The "undo file" format changed; if you see errors at startup during block
validation re-run with the -reindex flag to fix them.

Incompatible Changes
--------------------

This release no longer maintains a full index of historical transaction ids
by default, so looking up an arbitrary transaction using the getrawtransaction
RPC call will not work. If you need that functionality, you must run once
with -txindex=1 -reindex=1 to rebuild block-chain indices (see below for more
details).

Improvements
------------

Mac and Windows binaries are signed with certificates owned by the Bitcoin
Foundation, to be compatible with the new security features in OSX 10.8 and
Windows 8.

LevelDB, a fast, open-source, non-relational database from Google, is
now used to store transaction and block indices.  LevelDB works much better
on machines with slow I/O and is faster in general. Berkeley DB is now only
used for the wallet.dat file (public and private wallet keys and transactions
relevant to you).

Pieter Wuille implemented many optimizations to the way transactions are
verified, so a running, synchronized node uses much less memory and does
much less I/O. He also implemented parallel signature checking, so if you
have a multi-CPU machine all CPUs will be used to verify transactions.

New Features
------------

"Bloom filter" support in the network protocol for sending only relevant transactions to
lightweight clients.

contrib/verifysfbinaries is a shell-script to verify that the binary downloads
at sourceforge have not been tampered with. If you are able, you can help make
everybody's downloads more secure by running this occasionally to check PGP
signatures against download file checksums.

contrib/spendfrom is a python-language command-line utility that demonstrates
how to use the "raw transactions" JSON-RPC api to send coins received from particular
addresses (also known as "coin control").

New/changed settings (command-line or bitcoin.conf file)
--------------------------------------------------------

dbcache : now controls LevelDB memory usage. Running with (for example) -dbcache=1000
will use a gigabyte of memory and might make the initial blockchain download faster.

par : controls how many threads to use to validate transactions. Defaults to the number
of CPUs on your machine, use -par=1 to limit to a single CPU.

txindex : maintains an extra index of old, spent transaction ids so they will be found
by the getrawtransaction JSON-RPC method.

reindex : rebuild block and transaction indices from the downloaded block data.

New JSON-RPC API Features
-------------------------

lockunspent / listlockunspent allow locking transaction outputs for a period of time so
they will not be spent by other processes that might be accessing the same wallet.

addnode / getaddednodeinfo methods, to connect to specific peers without restarting.

importprivkey now takes an optional boolean parameter (default true) to control whether
or not to rescan the blockchain for transactions after importing a new private key.

Important Bug Fixes
-------------------

Privacy leak: the position of the "change" output in most transactions was not being
properly randomized, making network analysis of the transaction graph to identify
users' wallets easier.

Zero-confirmation transaction vulnerability: accepting zero-confirmation transactions
(transactions that have not yet been included in a block) from somebody you do not
trust is still not recommended, because there will always be ways for attackers to
double-spend zero-confirmation transactions. However, this release includes a bug
fix that makes it a little bit more difficult for attackers to double-spend a
certain type ("lockTime in the future") of zero-confirmation transaction.

Dependency Changes
------------------

Qt 4.8.3 (compiling against older versions of Qt 4 should continue to work)


Thanks to everybody who contributed to this release:
----------------------------------------------------

Alexander Kjeldaas
Andrey Alekseenko
Arnav Singh
Christian von Roques
Eric Lombrozo
Forrest Voight
Gavin Andresen
Gregory Maxwell
Jeff Garzik
Luke Dashjr
Matt Corallo
Mike Cassano
Mike Hearn
Peter Todd
Philip Kaufmann
Pieter Wuille
Richard Schwab
Robert Backhaus
Rune K. Svendsen
Sergio Demian Lerner
Wladimir J. van der Laan
burger2
default
fanquake
grimd34th
justmoon
redshark1802
tucenaber
xanatos


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: Scrat Acorns on February 09, 2013, 03:54:20 PM
0.8 syncs in just a few hours. It's great.

If you can test this RC please do so. The faster 0.8 is released the merrier.

Thank you for your hard work devteam.


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: nexusakachus on February 09, 2013, 04:01:57 PM
nice :)


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: ciphermonk on February 09, 2013, 04:06:34 PM
This is great news! 0.8 is a major milestone for the reference implementation! Thanks for all your dedicated work!


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: paraipan on February 09, 2013, 04:10:05 PM
Good work guys, keep it up!

Right now I would love to know how to code and contribute, but I'm just doing my part growing the bitcoin economy with small and sure steps. We're working hard here building the first Bitcoin exchange in Spain, and hope to have it up and running by the end of this month.


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: phatsphere on February 09, 2013, 05:04:25 PM
well, it is a fantastic improvement ... but still slow. since neither the disk is on the brink of dying, nor is the cpu above 5%, the network is the bottleneck.

i'm just curious, are there plans to speed up the data transfer rates? on weaker connections like mine, it will still take a long time to sync. this applies even more to less well connected 3rd world countries!

Is there some monitoring tool, that might also help to pinpoint problems with the networking (wireshark log?). looking into stream compression, or even other ways to compress based on references in the existing database ... and bundling up even more blocks in larger transfers is probably the only way to go?


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: Mike Hearn on February 09, 2013, 05:24:36 PM
Are you sure network is the bottleneck for you? Are you on an unusually slow connection? When syncing the chain at least, most nodes should be bottlenecked on disk seeks. So use iotop or something to look at that.

If network is the bottleneck then you probably want to switch from a full node to a lightweight node. The Bloom filtering that 0.8 provides can make them dramatically faster and use very little bandwidth.


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: jgarzik on February 09, 2013, 05:33:18 PM

My own notes:

  • Pieter gets a lions share of credit, for his ultraprune work
  • Please test!  Even an "it works for me" report is useful (just make sure to include your OS version and other helpful details in your "it works" post...)
  • If you still have a slow block download, simply close the program and restart.  That will start the download again, with another peer.  There are still known issues with peer selection.  "restart program" is an ugly but useful workaround.



Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: cypherdoc on February 09, 2013, 05:51:39 PM
i need those Windows binaries!


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: hazek on February 09, 2013, 05:51:51 PM
I'm going to sticky this for at least 14 days.


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: phatsphere on February 09, 2013, 05:59:32 PM
Are you sure network is the bottleneck for you? Are you on an unusually slow connection?
i used htop with the "r+w summary IO" column. there i get over a few seconds: 0, 0, ~150, ~27, 0, 0, ~150, ~20, 0, 0, ... for disk IO [edit: peak values could go up to over 1000]. I'm on a HDSPA mobile network, no port to the outside, and using a VPN w/o port forwarding. Yes, It's fine for browsing, but it has a unusual long latency.

But I'm not in a hurry, don't worry. I just did this including the VPN on purpose to test the networking speed. It's in my eyes (as an coder and admin but not a bitcoin dev.) definitely something which could be optimized. lightweight nodes are great, yes, but if you want to run your own node in a far-off country, you likely don't want it :-)

edit: watched the IO activity a bit longer. indeed, when there are somehow "harder" blocks, the IO goes up to 5000 and more -- and then still falls back to 0 for 1-2 seconds. interesting, so it might be IO, just not so obvious as before :-)


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: hazek on February 09, 2013, 06:10:48 PM
Btw are there any plans for adding the option to enter a username and password for a proxy?


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: Stapleddiet on February 09, 2013, 06:30:12 PM

My own notes:

  • Please test!  Even an "it works for me" report is useful (just make sure to include your OS version and other helpful details in your "it works" post...)



Please keep repeating that, I forget that it helps you folk who are developing to know whats going on.
I only start my wallet to keep updated with the blockchain every other day, I am in Australia in an area where expensive mobile data is my only option for the net, I feel guilty when I dont seed things but hey thanks to all the folk who can.
Win7, ATI 6950 downgraded from 5 cards in my mining days. I salute you who are developing BTC.


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: evoorhees on February 09, 2013, 07:29:24 PM
Nice! Thank you amazing Dev Team. You fill my dreams with beautiful visions of the future, and my waking hours with endless opportunity.

I've said it before and I'll say it again, the Dev Team is at the core of the most important project occurring anywhere on the planet. Downloading now.


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: oOoOo on February 09, 2013, 07:40:29 PM
1. Can we now delete the legacy blk000x.dat behemoth files?

2. crashes on OSX 10.6.8 at shutdown:

Code:
Exception Type:  EXC_BAD_ACCESS (SIGBUS)
Exception Codes: KERN_PROTECTION_FAILURE at 0x0000000000000000

Thread 4 Crashed:
0   com.yourcompany.Bitcoin-Qt    0x0023e963 CLevelDB::WriteBatch(CLevelDBBatch&, bool) + 35
1   com.yourcompany.Bitcoin-Qt    0x0023fc86 CBlockTreeDB::WriteBlockFileInfo(int, CBlockFileInfo const&) + 102
2   com.yourcompany.Bitcoin-Qt    0x000a1dd1 FindBlockPos(CValidationState&, CDiskBlockPos&, unsigned int, unsigned int, unsigned long long, bool) + 913
3   com.yourcompany.Bitcoin-Qt    0x000a2474 CBlock::AcceptBlock(CValidationState&, CDiskBlockPos*) + 1092
4   com.yourcompany.Bitcoin-Qt    0x000a39e6 ProcessBlock(CValidationState&, CNode*, CBlock*, CDiskBlockPos*) + 2758
5   com.yourcompany.Bitcoin-Qt    0x000aa208 LoadExternalBlockFile(__sFILE*, CDiskBlockPos*) + 1384
6   com.yourcompany.Bitcoin-Qt    0x000f4930 ThreadImport(void*) + 208
7   com.yourcompany.Bitcoin-Qt    0x0006b9fa boost::detail::thread_data<boost::_bi::bind_t<void, void (*)(void*), boost::_bi::list1<boost::_bi::value<void*> > > >::run() + 42
8   libboost_thread-mt.dylib      0x00b52845 boost::detail::thread_data_base::~thread_data_base() + 779
9   libSystem.B.dylib              0x96ee8259 _pthread_start + 345
10  libSystem.B.dylib 

.


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: foo on February 09, 2013, 10:14:41 PM
What's the disk space requirement of LevelDB vs the old BDB? (In percent.)


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: sega01 on February 09, 2013, 10:46:04 PM
Awesome! I'm trying to build 0.8.0rc1, but am having some problems.

Haven't looked into this a whole lot, but it looks like there's a bug with my gcc version, its usage, or how the headers are called out to. I can build up to the leveldb folder, after which it quits saying it can't find db/builder.h, which exists relative to the directory g++ is ran in. stracing g++ shows it never tries to access builder.h. I know little about C++ at all, but if I strip out the db/ in the include"", it picks it up. So it looks like g++ is treating includes in double quotes to be relative to the .cc file and not the CWD that gcc is called from.

Not sure where the bug is in exactly. Let me know if you would like any version information or help reproducing. This is with GCC 4.7.2.

Thanks,
Teran


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: dansmith on February 09, 2013, 11:17:29 PM
Gavin, there's a typo

Quote
1. Subdirectories in the data directory changed names; to avoid re-indexing
the blockchain, rename:
  mv $DATADIR/blktree $DATADIR/blocks/index
  mv $DATADIR/coins $DATADIR/chainstate

You actually want the content of blktree end up in blocks/index, so it should read
Quote
mv $DATADIR/blktree/* $DATADIR/blocks/index


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: jgarzik on February 09, 2013, 11:58:58 PM
Gavin, there's a typo

Quote
1. Subdirectories in the data directory changed names; to avoid re-indexing
the blockchain, rename:
  mv $DATADIR/blktree $DATADIR/blocks/index
  mv $DATADIR/coins $DATADIR/chainstate

You actually want the content of blktree end up in blocks/index, so it should read
Quote
mv $DATADIR/blktree/* $DATADIR/blocks/index

Strictly speaking, it is not a typo; $DATADIR/blocks simply has to exist first, in order for it to work.

So the instructions need "mkdir $DATADIR/blocks"



Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: Pieter Wuille on February 10, 2013, 12:32:39 AM
i'm just curious, are there plans to speed up the data transfer rates? on weaker connections like mine, it will still take a long time to sync. this applies even more to less well connected 3rd world countries!

There are known problems with the current  block synchronisation mechanism. Not exactly a bug, but just a crappy implementation. It's certainly intended to be changed, but it's not a trivial thing to do. Using the bootstrap.dat torrent, or using -connect=<ip> to a known fast node (not necessarily a trusted one) can speed up the initial download a lot. Not really nice solutions, but for now, it's all we have.

Quote
looking into stream compression, or even other ways to compress based on references in the existing database ... and bundling up even more blocks in larger transfers is probably the only way to go?

Block compression is certainly possible, but it's hard to get significant improvements. With a lot of effort and a very specifically designed compressed format, I expect we may get to something like 40%. Unfortunately, the blocks contain lots of hashes and cryptographic signatures, which are essentially uncompressible. Right now, I don't think these are a priority. Fixing the download mechanism will have a much larger impact.

1. Can we now delete the legacy blk000x.dat behemoth files?

Yes, you can if you don't intend to downgrade to 0.7.x (or below) code anymore. Note that the old and the new block files are hardlinked on supported platforms/filesystems, so you won't actually gain any space by deleting them. You can however delete blkindex.dat.

What's the disk space requirement of LevelDB vs the old BDB? (In percent.)

There's no simple answer to this, as it's not just a change from BDB to LevelDB: the actual layout and organisation of the databases changed as well. The actual blocks haven't changed (they are in different, more and smaller files, but the data is the same). The old blkindex.dat (BDB, ~1650 MiB) was replaced by:
  • the block index (LevelDB, $DATADIR/blocks/index, ~ 30 MiB)
  • the chain state (LevelDB, $DATADIR/chainstate, ~ 170 MiB)
  • the undo data (custom format, $DATADIR/blocks/rev*.dat, ~ 700 MiB)
So, roughly, the old index data now takes half as much space, but it's not entirely a fair comparison (we actually do store less information now).


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: wtfvanity on February 10, 2013, 12:42:51 AM
Quote
This release no longer maintains a full index of historical transaction ids
by default, so looking up an arbitrary transaction using the getrawtransaction
RPC call will not work. If you need that functionality, you must run once
with -txindex=1 -reindex=1 to rebuild block-chain indices (see below for more
details).

Quick question regarding this. Does importprivkey still work? It rescans the block chain to see if it has been used, would this still work with the way it stores the historical transaction ids? Or would I need to use the txindex=1 if I plan on importing keys?


I've tested on a couple of set up of windows:
Windows XP New install, download block chain: Works okay. Much faster than previous versions on downloading.
Windows 7 New install, download block chain: Works okay. Much faster than previous versions on downloading.

Windows XP block chain was at about 190,000 never finished downloading because it took too long for what this used to be used for. Took about an hour minutes to reindex what it had then it downloaded the rest nice and quickly. This was an older P4 machine.

Windows 7, had full block chain. It reindexed fine nice and fast in about an hour. I did notice during the first 3/4 of the block chain it didn't really use extra CPU. It's near the end when it started to crank up the engine. This machine was a more modern machine multiple cores and I set the dbcache to 2000.


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: Pieter Wuille on February 10, 2013, 12:49:32 AM
Quick question regarding this. Does importprivkey still work? It rescans the block chain to see if it has been used, would this still work with the way it stores the historical transaction ids? Or would I need to use the txindex=1 if I plan on importing keys?

No problem - rescan just goes through all old blocks again, one by one. Those are still available in 0.8. The only thing that changes is that no information in the index is kept about spent transactions, so without txindex, it's impossible to find a transaction given just its txid. In practice, really the only thing affected is the getrawtransaction RPC.


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: wtfvanity on February 10, 2013, 12:54:56 AM
Quick question regarding this. Does importprivkey still work? It rescans the block chain to see if it has been used, would this still work with the way it stores the historical transaction ids? Or would I need to use the txindex=1 if I plan on importing keys?

No problem - rescan just goes through all old blocks again, one by one. Those are still available in 0.8. The only thing that changes is that no information in the index is kept about spent transactions, so without txindex, it's impossible to find a transaction given just its txid. In practice, really the only thing affected is the getrawtransaction RPC.


Oh duh, I knew that. I've read what we are doing here and also about light nodes. I mistook the spent tx as creation of a light node. My mistake. I understand now. The bloom feature is groundwork for a light node. I'm going to test on another windows machine and a linux machine later. But so far I love what I see.

This really opens up the, hey, download bitcoin and try it out I'll send you some coins instead of; hey, check out bitcoin, but get an alternate client because of the official one takes too long.


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: rini17 on February 10, 2013, 12:55:06 AM
I ran bitcoind 0.8rc provided 64bit linux binary (with -dbcache=1000), it synced in just 1 hour using single connection to locally running bitcoind 0.7, while being responsive to RPC queries all the time. I like it 8)


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: Andreas Schildbach on February 10, 2013, 09:05:05 AM
It works for me, self-compiled bitcoin-qt v0.8.0-rc1 from git on Ubuntu 12.10 64bit.

I tested migration by re-indexing once and also completely downloading the block chain once. Also a couple of payments both directions.

Specifically, tested Bloom Filtering a lot, in combination with Bitcoin Wallet for Android (bloom enabled previews can be downloaded here (http://code.google.com/p/bitcoin-wallet/downloads/list)).

I can say that bitcoin-qt is a lot faster, I'd even say it jumped from unusable to very well usable. Good work everyone!


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: LightRider on February 10, 2013, 10:03:58 AM
Successfully updated on Win7x64. Reindexing took a couple of hours, and the last 5000 blocks or so pegged the cpu. Also brought up testnet from scratch in under 12 minutes. Well done everyone!


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: Akka on February 10, 2013, 10:16:59 AM

Please test!  Even an "it works for me" report is useful (just make sure to include your OS version and other helpful details in your "it works" post...)

In the spirit of this:

Deleted complete blockchain, upgraded: Blockchain downloaded in under 4 Hours

Send / receive works fine.

No Bugs seen so far.

Awesome Work.

OS: Win 7

Edit:

Testers:

Another good, simple test:  leave it running for 24+ hours, especially on Windows.

This helps verify longer term stability, and verifies that it keeps up with the network.

There had been some crash reports early in development, on Windows.  Pretty sure those are solved, but extra testing through normal use cannot hurt.

My PC runs 24/7. I will post if I have any crashes. (None so far)


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: hazek on February 10, 2013, 10:29:44 AM
Successfully updated on Win7x64. Reindexing took a 7 hours or so, did it with -dbcache=2000, after it was finished I closed it without a problem and I have since ran it again without that command and it works. I haven't tried sending or receiving anything.


I'd still like to get an answer when, if at all, will there be an option to enter a username and password for a proxy server?


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: jgarzik on February 10, 2013, 10:35:24 AM

Testers:

Another good, simple test:  leave it running for 24+ hours, especially on Windows.

This helps verify longer term stability, and verifies that it keeps up with the network.

There had been some crash reports early in development, on Windows.  Pretty sure those are solved, but extra testing through normal use cannot hurt.



Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: Technomage on February 10, 2013, 12:33:22 PM
My PC: Windows 7 with an old Intel Quad-Core. 4 GB memory. 7200 RPM regular hard drive. Decent computer.

First impressions: the reindexing is fairly fast. I've now been at it for a little more than 1,5 hours and there are only 2800 blocks left. However, at around the 5000 blocks left mark, my PC started lagging big time. I checked that CPU usage (with a quad-core OC'd to 3 ghz) was pretty much 100% and there was 0 memory free. Low usability of the computer, barely usable.

That's it for now, I will keep it running for a good while to see how it works.

Edit: The last few thousand blocks went fairly quickly as well. Total reindex took 2hrs approx. Now I'll just keep it running.


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: jonathan on February 10, 2013, 01:26:13 PM
bitcoind 0.8.0rc1 64bit on debian squeeze. Started a fresh download of the blockchain, came back five hours later, it was downloaded up to the present block. No hitches to report. Sweet!


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: Raoul Duke on February 10, 2013, 02:25:39 PM
The *nix x64 binary works wonderfully on Ubuntu 12.04 64bits.
It even solved the status bar icon/menus not showing problem.

Downloaded and verified the complete blockchain in less than 5 hours on a 2.2GHz Pentium dual-core laptop with 4GB RAM.


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: wtfvanity on February 10, 2013, 03:22:55 PM
All four of the first machines are running just fine.

I did another test on a more recent CPU plenty of ram running Windows 7. I let it run overnight and download the whole blockchain. This morning I ran windows updates that were pending and prepared for a reboot. The block chain had not finished downloading all of the way yet (I used add IP from a local node for all these, probably why my results were faster than most others posted here)

I did a file exit on Bitcoin-qt and got this exception:

Code:
A fatal error occurred. Bitcoin can no longer continue safely and will quit.

Exception: St9pad_alloc
std:bad_alloc
C:\Program Files\Bitcoin\bitcoin-qt.exe in Runaway exception

These are the last entries in the debug log

Code:
SetBestChain: new best=000000000000018829028b3af6d47eac9dab3f4b233c77aff1ee2262fcd0faaa  height=214069  work=687730958631335352866  tx=10451422  date=2012-12-28 19:24:59
SetBestChain: new best=000000000000002aeab1c5b9fc27e14f759ab16a3e4cde600d98718b93e50416  height=214070  work=687743756268435510981  tx=10451918  date=2012-12-28 19:35:21
SetBestChain: new best=000000000000057d76f8d7a62f3de9a874055874614c6423fce82f60b0177d9a  height=214071  work=687756553905535669096  tx=10452418  date=2012-12-28 19:40:47

I opened Bitcoin back up and it pegged all cores of the CPU for a while. I had some stuff minimizing and maximizing so I'm not sure what the splash screen said that it was doing. This was much longer by a significant amount vs. the other machines when I close Bitcoin and open them back up. In fact, it is still not open now after about 10 minutes and after it opens I'll post more details. But for now, there is a crash on RC1 with Windows 7. It did not crash until I asked it to shut down. I checked the resource usage, Bitcoin-qt was using about a gig of ram, machine has 8 GB and wasn't even at 50% utilization for ram.



Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: hxtop on February 10, 2013, 03:31:53 PM
nice work


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: kalleguld on February 10, 2013, 03:47:46 PM
Running great here.
Linux nas1 2.6.33-rc6-00219-gf5c72e7-svn #8 PREEMPT Tue Feb 9 16:38:20 CET 2010 armv5tel GNU/Linux

Wanted to setup an electrum server, did not notice that I downloaded the 0.8.0 source until I recognized the dbcache parameter in this topic.

tl;dr bitcoind with electrum-patch runs fine on armv5 and old kernel


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: mmortal03 on February 10, 2013, 03:50:31 PM
Am testing it now, completely fresh install on a netbook, Windows 8 64-bit, AMD E-350, 6 GB of RAM, 5400 rpm hard drive, using -dbcache=2000.

Will report back once it's done.


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: wtfvanity on February 10, 2013, 04:20:39 PM
All four of the first machines are running just fine.

I did another test on a more recent CPU plenty of ram running Windows 7. I let it run overnight and download the whole blockchain. This morning I ran windows updates that were pending and prepared for a reboot. The block chain had not finished downloading all of the way yet (I used add IP from a local node for all these, probably why my results were faster than most others posted here)

I did a file exit on Bitcoin-qt and got this exception:

Code:
A fatal error occurred. Bitcoin can no longer continue safely and will quit.

Exception: St9pad_alloc
std:bad_alloc
C:\Program Files\Bitcoin\bitcoin-qt.exe in Runaway exception

These are the last entries in the debug log

Code:
SetBestChain: new best=000000000000018829028b3af6d47eac9dab3f4b233c77aff1ee2262fcd0faaa  height=214069  work=687730958631335352866  tx=10451422  date=2012-12-28 19:24:59
SetBestChain: new best=000000000000002aeab1c5b9fc27e14f759ab16a3e4cde600d98718b93e50416  height=214070  work=687743756268435510981  tx=10451918  date=2012-12-28 19:35:21
SetBestChain: new best=000000000000057d76f8d7a62f3de9a874055874614c6423fce82f60b0177d9a  height=214071  work=687756553905535669096  tx=10452418  date=2012-12-28 19:40:47

I opened Bitcoin back up and it pegged all cores of the CPU for a while. I had some stuff minimizing and maximizing so I'm not sure what the splash screen said that it was doing. This was much longer by a significant amount vs. the other machines when I close Bitcoin and open them back up. In fact, it is still not open now after about 10 minutes and after it opens I'll post more details. But for now, there is a crash on RC1 with Windows 7. It did not crash until I asked it to shut down. I checked the resource usage, Bitcoin-qt was using about a gig of ram, machine has 8 GB and wasn't even at 50% utilization for ram.



45 minutes later it starts up and finishes the block chain very quickly. I closed it down and opened it back up just fine nice and fast. Not sure what other information I can provide about this but it seems that everything is okay now.


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: jl2012 on February 10, 2013, 04:46:53 PM
Does it work with Armory?


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: Gavin Andresen on February 10, 2013, 06:06:42 PM
The crash-on-exit bug could be this issue:  https://github.com/bitcoin/bitcoin/issues/2204

We may have to live with it for the 0.8 release, and fix it in the next release, because there is a high risk that fixing it will cause more problems than it solves.



Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: xxjs on February 10, 2013, 06:35:48 PM
Ubuntu 12.04 64bit I7 12GB
Empty wallet behind Armory 0.87-beta

Works great. CPU load while reindexing blocks is 600%, around 10 MB/s disk io. Fans revv up. Nice to put all the horses before the cart.


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: Diapolo on February 10, 2013, 06:49:43 PM
It even solved the status bar icon/menus not showing problem.

I remember there was an open issue on Github, was that yours? Perhaps you can add that update there and close the ticket.

Thanks,
Dia


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: Raoul Duke on February 10, 2013, 06:54:10 PM
It even solved the status bar icon/menus not showing problem.

I remember there was an open issue on Github, was that yours? Perhaps you can add that update there and close the ticket.

Thanks,
Dia

I thought that was solved. It seems it isn't. Just now opened it again and the icon and menus are gone, despite being there the previous 5 or 6 times I started Bitcoin-Qt 0.8.0-RC1 since I installed it.


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: Diapolo on February 10, 2013, 06:55:53 PM
It even solved the status bar icon/menus not showing problem.

I remember there was an open issue on Github, was that yours? Perhaps you can add that update there and close the ticket.

Thanks,
Dia

I thought that was solved. It seems it isn't. Just now opened it again and the icon and menus are gone, despite being there the previous 5 or 6 times I started Bitcoin-Qt 0.8.0-RC1 since I installed it.


Too bad ... seems to be a Qt or OS UI issue though.

Dia


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: foo on February 10, 2013, 07:54:28 PM
Upgraded from 0.7.1 on Windows XP - smooth sailing so far. Syncing is now CPU-limited with very little thrashing of the hard drive, nice work!


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: mmortal03 on February 10, 2013, 08:23:13 PM
Am testing it now, completely fresh install on a netbook, Windows 8 64-bit, AMD E-350, 6 GB of RAM, 5400 rpm hard drive, using -dbcache=2000.

Will report back once it's done.

Bummer, midway through downloading the blockchain, with -dbcache=2000, it kept using up RAM all the way to 4GB and then crashed.

On restart, it sits at the wallet picture with "xporting blocks from block databas" (text cut off on left and right hand side).  I can see from the Task Manager that it's doing something, pegged CPU at 50%, and increasing RAM. I'll let it sit and see where it goes.

Edit: It finally came out of the splash screen and CPU usage dropped to normal, but RAM is still increasing. I've got it set to -dbcache=1000 this time, so we'll see if it sticks to that.


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: wtfvanity on February 10, 2013, 09:55:02 PM
The crash-on-exit bug could be this issue:  https://github.com/bitcoin/bitcoin/issues/2204

We may have to live with it for the 0.8 release, and fix it in the next release, because there is a high risk that fixing it will cause more problems than it solves.



Not sure if this is exactly what happened, but it only did it on one machine. After it started back up, it is now working fine. I closed it after it caught up to the block chain and no error. I closed out of one other instance of bitcoin and it didn't error. Opened it back up. I'm going to leave one running and the other three I'm going to close every 24 hours or so and see if it opens up again. Besides giving it extra work the first time it closed, it didn't look like it really had any other side effects.

If there is anything I can do to help squash it or test it, let me know. The ones that don't reproduce consistently are difficult. This machine has identical specs as the other Win 7 machine and the first one didn't do it. Not sure.


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: inglian on February 11, 2013, 01:08:45 AM
All good with bitcoin-0.8.0rc1-win32
  • Windows 7
  • HP 8440 laptop with 4 cores
  • 8GB RAM
  • FiOS connection

Good upgrade from 0.7.1-353-g3afefd8-win32

With an empty data directory and bootstrap.dat, 3h 40m to current blockheight. Multithreading doesn't seem to start until after bootstrap.dat is loaded.

Good upgrade from 0.7.2. Reindexing completed through blockheight 216274 in 37m. Update to current block from the network had long periods with no data coming in, but did show good multithreading when data was arriving.


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: Grami on February 11, 2013, 06:34:13 AM
Ubuntu 12.10 64bit. Bitcoin-qt crashed on initial blockchain download (process killed). After program restart - main window doesnt show. After system reboot - works fine.


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: deepceleron on February 11, 2013, 07:09:10 AM
The blockchain checkpoint in this release has been updated to 216116. I created a bootstrap.dat of this height and successfully imported it in 37 minutes with bitcoin-qt on Win7 64 bit. This bug appears quashed: https://github.com/bitcoin/bitcoin/issues/1951

Using one of the fastest SSDs made, Bitcoin is CPU-limited and thread-limited, holding one CPU core at 100%, and blipping a second core to ~75% every few seconds.

02/10/2013  08:44 PM     4,855,459,871 bootstrap.dat
SHA256: bf658c7055b733bfc15ea167f298c5599b89d220b14dbe7c8ef20b18e468c451 *bootstrap.dat

Code:
2013-02-11 06:35:35 SetBestChain: new best=000000000000044263dc2253bdd2a31a4194ae13321b25e41f422e148c417cec  height=216113  work=714355361457244290318  tx=11010043  date=2013-01-11 10:45:37
2013-02-11 06:35:35 ProcessBlock: ACCEPTED
2013-02-11 06:35:35 SetBestChain: new best=0000000000000234440faa3ee1a84b457e5f79515a2b19853d7e72422e3183f0  height=216114  work=714369318379402124429  tx=11010572  date=2013-01-11 10:56:20
2013-02-11 06:35:35 ProcessBlock: ACCEPTED
2013-02-11 06:35:35 SetBestChain: new best=00000000000000f97afdf8ccba49919bb998313ec67e3654798b86a3f6631c1e  height=216115  work=714383275301559958540  tx=11010714  date=2013-01-11 10:59:18
2013-02-11 06:35:35 ProcessBlock: ACCEPTED
2013-02-11 06:35:35 connection timeout
2013-02-11 06:35:36 SetBestChain: new best=00000000000001b4f4b433e81ee46494af945cf96014816a4e2370f11b23df4e  height=216116  work=714397232223717792651  tx=11011160  date=2013-01-11 11:11:30
2013-02-11 06:35:36 ProcessBlock: ACCEPTED
2013-02-11 06:35:36 Loaded 216116 blocks from external file in 2218342ms

I will offer this as a torrent and also give direct-download links to those who wish to jump-start and seed such a torrent, since my seeding speed is limited. The question is, should an official torrent going forward be a compressed file? Using the highest compression preset with xz (http://tukaani.org/xz/)/lzma (installed with most distros, opens with 7-zip or MacOS GUI tools (http://unarchiver.c3.cx/formats)) saves 2GB of downloading, so I say yes:

02/11/2013  01:09 AM     2,780,285,148 bootstrap.dat.xz

Also, bootstrap.dat is too big for FAT32, but not the compressed version. Bitcoin 0.9+ could even (feature request) open a compressed bootstrap (lzma is a stream container) directly if standardized.


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: jgarzik on February 11, 2013, 07:54:36 AM
I will offer this as a torrent and also give direct-download links to those who wish to jump-start and seed such a torrent, since my seeding speed is limited. The question is, should an official torrent going forward be a compressed file? Using the highest compression preset with xz (http://tukaani.org/xz/)/lzma (installed with most distros, opens with 7-zip or MacOS GUI tools (http://unarchiver.c3.cx/formats)) saves 2GB of downloading, so I say yes:

02/10/2013  08:44 PM     2,841,903,993 bootstrap.dat.lzma

Also, bootstrap.dat is too big for FAT32, but not the compressed version. Bitcoin 0.9+ could even (feature request) open a compressed bootstrap (lzma is a stream container) directly if standardized.

This would make a useful contribution to this thread,

   [BETA] Bitcoin blockchain torrent
   https://bitcointalk.org/index.php?topic=117982.0

The semi-official bootstrap.dat torrent is about to be updated, as it is for each bitcoin release.  The 2GB issue has already been raised in that thread, and the FAT32 4GB issue would also be a good one to raise.

Ideally we would like everyone to seed the same torrent -- it's the same data, as guaranteed by bitcoin (as well as torrent) hashes.



Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: caveden on February 11, 2013, 08:05:23 AM
Thank you all for this development. This version is a major one for scalability, with bloom filters and the pruned index.

However, this release includes a bug
fix that makes it a little bit more difficult for attackers to double-spend a
certain type ("lockTime in the future") of zero-confirmation transaction.

Could somebody point me to some clarification on what exactly does this means?
Because, AFAIK, time locked transactions should be reversible, up until the block they are supposed to be included. That's the whole point of time locking, isn't it?


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: Mike Hearn on February 11, 2013, 08:28:46 AM
Yes that's how they're supposed to work. However that functionality is disabled.


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: jonathan on February 11, 2013, 08:33:37 AM
reporting back in from a debian squeeze box running bitcoind 0.8.0rc1-linux 64bit.

bitcoind has been running fine for over 24 hours. I just closed it down then started it again. All working, fast and smooth. No problems to report here.


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: caveden on February 11, 2013, 08:39:32 AM
Yes that's how they're supposed to work. However that functionality is disabled.

What does "disabled" means?
Does it means that bitcoind gives you no way to deal with nLockTime, but if some miner accepts, relays and mines them, his blocks will be accepted?


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: jgarzik on February 11, 2013, 09:02:18 AM
Yes that's how they're supposed to work. However that functionality is disabled.

What does "disabled" means?
Does it means that bitcoind gives you no way to deal with nLockTime, but if some miner accepts, relays and mines them, his blocks will be accepted?

Blocks only contain "final" transactions, and by definition, a transaction with non-zero nLockTime is not final.  Transaction replacement occurs before miners mine blocks.

Thus, to answer your question, such a miner's blocks would be rejected as invalid by all existing versions of bitcoin software.


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: caveden on February 11, 2013, 09:10:33 AM
Blocks only contain "final" transactions, and by definition, a transaction with non-zero nLockTime is not final.  Transaction replacement occurs before miners mine blocks.

Thus, to answer your question, such a miner's blocks would be rejected as invalid by all existing versions of bitcoin software.

That was not exactly my question (I meant time-locked tx after their time locking expires) but never mind, I got an answer in the other thread. Thanks,


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: phatsphere on February 11, 2013, 11:15:30 AM
hi, i did some unethical "stress testing". Deleted all the data, started to re-download everything. after some time, i repeatedly killed it via pkill -9 bitcoin. well, that worked fine.

then I started to delete some random files and now it no longer loads!

What I would expect is to detect a corrupt database, some re-indexing action, or just a nice notice that it has to redo everything due to corruption. instead:

a box "error initializing block database" -> OK -> core dumped :-(

the random files i deleted:
Code:
rm ./blocks/index/000189.sst ./chainstate/000012.sst

debug.log:

Code:
Bitcoin version v0.8.0rc1-beta (2013-02-06 16:06:43 -0500)
Using OpenSSL version OpenSSL 0.9.8k 25 Mar 2009
Startup time: 2013-02-11 11:12:57
Default data directory /users/XXX/.bitcoin
Used data directory /users/XXX/.bitcoin
Using 2 threads for script verification
init message: Verifying wallet integrity...
dbenv.open LogDir=/users/XXX/.bitcoin/database ErrorFile=/users/XXX/.bitcoin/db.log
Bound to [::]:8333
Bound to 0.0.0.0:8333
init message: Loading block index...
Opening LevelDB in /users/XXX/.bitcoin/blocks/index
Opened LevelDB successfully
Opening LevelDB in /users/XXX/.bitcoin/chainstate
Opened LevelDB successfully
LoadBlockIndex(): last block file = 0
LoadBlockIndex(): last block file: CBlockFileInfo(blocks=86483, size=37352096, heights=0..86479, time=2009-01-03..2010-10-20)
LoadBlockIndex(): transaction index disabled
LoadBlockIndex(): hashBestChain=00000000001aa62850aea3dda34b3a9af269a0cac44c3ea481206bdfdf5dfd91  height=79504 date=2010-09-13 13:05:59
Initializing databases...
000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f
000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f
4a5e1e4baab89f3a32518a88c31bc87f618f76673e2cc77ab2127b7afdeda33b
CBlock(hash=000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f, ver=1, hashPrevBlock=0000000000000000000000000000000000000000000000000000000000000000, hashMerkleRoot=4a5e1e4baa, nTime=1231006505, nBits=1d00ffff, nNonce=2083236893, vtx=1)
  CTransaction(hash=4a5e1e4baa, ver=1, vin.size=1, vout.size=1, nLockTime=0)
    CTxIn(COutPoint(0000000000, 4294967295), coinbase 04ffff001d0104455468652054696d65732030332f4a616e2f32303039204368616e63656c6c6f72206f6e206272696e6b206f66207365636f6e64206261696c6f757420666f722062616e6b73)
    CTxOut(nValue=50.00000000, scriptPubKey=04678afdb0fe5548271967f1a67130)
  vMerkleTree: 4a5e1e4baa
ERROR: AddToBlockIndex() : 000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f already exists
ERROR: LoadBlockIndex() : genesis block not accepted

PS: relevant files that are still alive:

Code:
./chainstate/000022.log
./chainstate/000015.sst
./chainstate/MANIFEST-000021
./chainstate/CURRENT
./chainstate/LOCK
./chainstate/000013.sst
./chainstate/LOG
./chainstate/LOG.old
./blocks/rev00000.dat
./blocks/blk00000.dat
./blocks/index/MANIFEST-000221
./blocks/index/000188.sst
./blocks/index/000222.sst
./blocks/index/000223.log
./blocks/index/000191.sst
./blocks/index/000216.sst
./blocks/index/000219.sst
./blocks/index/CURRENT
./blocks/index/LOCK
./blocks/index/000224.sst
./blocks/index/000190.sst
./blocks/index/000187.sst
./blocks/index/000211.sst
./blocks/index/LOG
./blocks/index/000213.sst
./blocks/index/LOG.old

PPS: and after deleting everything, I also had a corrupted wallet. (nothing lost, don't worry, but just let you know that for some reason, it got corrupted during that  treatment! likely (?) due to the forced shutdown while it was still open). and yes, it was fine before, and it's size was around 100k before and afterwards.


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: mmortal03 on February 11, 2013, 11:25:57 AM
Am testing it now, completely fresh install on a netbook, Windows 8 64-bit, AMD E-350, 6 GB of RAM, 5400 rpm hard drive, using -dbcache=2000.

Will report back once it's done.

Bummer, midway through downloading the blockchain, with -dbcache=2000, it kept using up RAM all the way to 4GB and then crashed.

On restart, it sits at the wallet picture with "xporting blocks from block databas" (text cut off on left and right hand side).  I can see from the Task Manager that it's doing something, pegged CPU at 50%, and increasing RAM. I'll let it sit and see where it goes.

Edit: It finally came out of the splash screen and CPU usage dropped to normal, but RAM is still increasing. I've got it set to -dbcache=1000 this time, so we'll see if it sticks to that.

I closed the instance running -dbcache=1000 to see how it would handle it, and it closed no problem. I've restarted it with -dbcache=1500, and have let it run for a while. Memory use is now up to 1,660,316 KB, so, similar to my results using -dbcache=2000 where it went up to 4GB, it doesn't seem to be minding the value chosen in this setting as a limit.


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: hazek on February 11, 2013, 11:30:57 AM
Successfully updated on Win7x64. Reindexing took a 7 hours or so, did it with -dbcache=2000, after it was finished I closed it without a problem and I have since ran it again without that command and it works. I haven't tried sending or receiving anything.


I'd still like to get an answer when, if at all, will there be an option to enter a username and password for a proxy server?


Running now for over 36hours without a problem.


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: Technomage on February 11, 2013, 11:39:24 AM
I've been running it for a good while as well. Many sent and received transactions also. No problems whatsoever.


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: btcven on February 11, 2013, 11:48:42 AM
I only connect to 8 nodes, even restarting the client.

Have been hours running, still downloading the blockchain (really slow internet connection).

Problems: closing the app window is not enough, must go to Dock and Quit the app.

MacBook Air, 128GB SSD, 10.8.2


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: neuronics on February 11, 2013, 02:09:12 PM
Everything working like expected on Win7 32 and XP 64 machines / v0.8.0rc1-beta / Armory 0.87 beta, reindexing blocks ~75 minutes.  Very impressing !  Good job, thx alot.


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: Ente on February 11, 2013, 03:23:08 PM
The whole pruning and bloom feature is a bit unclear to me.
So I basically send tx to other nodes when they fit their bloom filter?
But I still have the whole blockchain locally stored?

Where my question aims at:
I run a bitcoind on my desktopmachine, connecting to it via Armory when needed. I wish to have a "full" node, saving and serving every tx in the blockchain.
Also, I wish to connect to that bitcoind via Android Schildbach Client from time to time, for fast catch-up.
And, you know, the network would not work if *every* node was a light client, right?

There must be a place where these kind of questions are already answered, sorry for asking here.

Can't wait to upgrade once I get home!

Ente


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: caveden on February 11, 2013, 03:32:02 PM
The whole pruning and bloom feature is a bit unclear to me.
So I basically send tx to other nodes when they fit their bloom filter?
But I still have the whole blockchain locally stored?

These are two unrelated things. Bloom filter is pretty much what you describe: a protocol for full nodes to serve lightweight nodes only the transactions that concern them, plus some false positives for privacy reasons.

The index pruning changed the main transaction index in bitcoind, now it only stores unspent transactions, what makes it much smaller, and thus synchronizing with the network now requires much less I/O. Keep in mind this is just the index, the full blockchain content is still stored.


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: deepceleron on February 11, 2013, 03:55:10 PM
hi, i did some unethical "stress testing". Deleted all the data, started to re-download everything. after some time, i repeatedly killed it via pkill -9 bitcoin. well, that worked fine.
...
PPS: and after deleting everything, I also had a corrupted wallet. (nothing lost, don't worry, but just let you know that for some reason, it got corrupted during that  treatment! likely (?) due to the forced shutdown while it was still open). and yes, it was fine before, and it's size was around 100k before and afterwards.

Corrupting a wallet is almost expected eventually if you continue to kill Bitcoin while downloading blocks. The last-seen block hash is written to the wallet every block, which means the wallet is sometimes updated dozens of times a second while downloading blocks that fast.

Did Bitcoin attempt to automatically salvage the wallet, and was this successful? I noted and filed an issue where you continue to get warning messages every launch after a wallet salvage was performed.


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: Rygon on February 11, 2013, 04:06:21 PM
hi, i did some unethical "stress testing". Deleted all the data, started to re-download everything. after some time, i repeatedly killed it via pkill -9 bitcoin. well, that worked fine.
...
PPS: and after deleting everything, I also had a corrupted wallet. (nothing lost, don't worry, but just let you know that for some reason, it got corrupted during that  treatment! likely (?) due to the forced shutdown while it was still open). and yes, it was fine before, and it's size was around 100k before and afterwards.

Corrupting a wallet is almost expected eventually if you continue to kill Bitcoin while downloading blocks. The last-seen block hash is written to the wallet every block, which means the wallet is sometimes updated dozens of times a second while downloading blocks that fast.

Did Bitcoin attempt to automatically salvage the wallet, and was this successful? I noted and filed an issue where you continue to get warning messages every launch after a wallet salvage was performed.

Is there a way to safely shut down bitcoin-qt while downloading blocks without the risk of corrupting a wallet? Sometimes I just have to shut down the computer while waiting hours to download blocks, and the only way to stop Bitcon is to close the window in Ubuntu. Is that still safe?

Also, I've had a corrupted wallet file that Bitcoin was unable to salvage. No loss, but it was concerning.


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: Ente on February 11, 2013, 04:38:21 PM
The whole pruning and bloom feature is a bit unclear to me.
So I basically send tx to other nodes when they fit their bloom filter?
But I still have the whole blockchain locally stored?

These are two unrelated things. Bloom filter is pretty much what you describe: a protocol for full nodes to serve lightweight nodes only the transactions that concern them, plus some false positives for privacy reasons.

The index pruning changed the main transaction index in bitcoind, now it only stores unspent transactions, what makes it much smaller, and thus synchronizing with the network now requires much less I/O. Keep in mind this is just the index, the full blockchain content is still stored.

Thank you for clarifying.
So with pruning, the whole block[chain] is still downloaded and served. Only a part of it is then used for the "user-side" of bitcoin, where it cares about what your addresses are and that?
Sounds great!

Ente


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: Mike Hearn on February 11, 2013, 04:45:40 PM
I run a bitcoind on my desktopmachine, connecting to it via Armory when needed. I wish to have a "full" node, saving and serving every tx in the blockchain.
Also, I wish to connect to that bitcoind via Android Schildbach Client from time to time, for fast catch-up.
And, you know, the network would not work if *every* node was a light client, right?

Your Android client will use Bloom filtering to reduce its bandwidth usage and speed things up (if you use the latest versions which are not released yet).


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: wtfvanity on February 11, 2013, 06:38:27 PM
Error upgrading on Ubuntu 11.04 server.

Quote
************************
EXCEPTION: St9bad_alloc
std::bad_alloc
bitcoin in ProcessMessages()



************************
EXCEPTION: St9bad_alloc
std::bad_alloc
bitcoin in ThreadMessageHandler()

terminate called after throwing an instance of 'std::bad_alloc'
  what():  std::bad_alloc
Aborted

It had the full block chain, was upgrading, not sure where it was in the blocks > 170k I know that for sure...


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: Pieter Wuille on February 11, 2013, 08:32:59 PM
So with pruning, the whole block[chain] is still downloaded and served. Only a part of it is then used for the "user-side" of bitcoin, where it cares about what your addresses are and that?

Not really. None of this has anything to do with the wallet implementation. Wallets always track all transactions relevant to the user. This has not (or hardly) changed between 0.7 and 0.8.

What has changed, is how the block and transaction validation works. Previously, we stored
  • the full blocks (blk000?.dat)
  • the (byte) position of every block and transaction in it (blkindex.dat)
  • for every transaction output, whether and where is was spent (also blkindex.dat)
This required an ever-growing index, and fast access to the full (ever growing) block data. This was slow.

The new system stores:
  • the full blocks (blocks/blk000??.dat)
  • the (byte) position of every block in it (but not every transaction!) (blocks/index/*)
  • a separate database with a copy of the unspent transaction outputs (so not an index with byte positions for into the block chain, just a copy of not even the full transactions, but only the part that may be relevant in the future) (chainstate/*)
  • an undo log for the chain state, so we can go back in time for reorganisations (blocks/rev000??.dat).
The big advantage is that we now only need fast access to the chain state (around 150 MB), instead of to the full blocks and the full index (several GB).

Although I initially called this new database/validation system "ultraprune". This is a very confusing name, as there is no actual pruning going on: we still keep all blocks/transactions around. The block data is still necessary for rescanning, reorganising, serving to other nodes, and the getrawtransaction RPC call. The code that resulted in the new database system actually started as an experiment about how small the chain state (aka UTXO set) could be represented, by pruning it as hard as possible. This is where the name comes from.


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: kgo on February 12, 2013, 12:55:17 AM
Upgraded a 64-bit system that was previously running next-test on the latest Ubuntu LTS.  Instructions provided about deleting files worked just fine.

Upgraded a 32-bit system running 0.7.3 on Debian Squeeze.  The conversion of the old blocks seemed to slow down near the end, but left things running overnight and everything worked.

Sent one bitcoin back and forth between systems without incident.


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: dooglus on February 12, 2013, 04:31:12 AM
I built the v0.8.0rc1 tag from git on ubuntu 12.10 64 bit and it works fine so far.

I missed this bit:

Quote
If you helped test pre-release versions, there are two changes that you
should be aware of:

and so it started downloading/importing (not sure which) the whole blockchain from scratch.  So I quit, ran the mkdir/mv commands from the OP, and restarted but it was too late.  The debug.log showed all the blocks past the first 500 as being orphaned:

Quote
SetBestChain: new best=00000000806df68baab17e49e567d4211177fef4849ffd8242d095c6a1169f45  height=499  work=2147516416500  tx=508  date=2009-01-14 21:14:40
ProcessBlock: ACCEPTED
SetBestChain: new best=000000004ff664bfa7d217f6df64c1627089061429408e1da5ef903b8f3c77db  height=500  work=2151811449333  tx=509  date=2009-01-14 21:27:29
ProcessBlock: ACCEPTED
ProcessBlock: ORPHAN BLOCK, prev=00000000046739c1ea3612322e1769f07783ebb22fb62498b7fd2ff249a52f29
ProcessBlock: ORPHAN BLOCK, prev=000000000bbb0dde89c4ccd3d5faced4cedb506bd8a74a74db09558e7df55959

and then, after complaining about 220k orphaned blocks started downloading from block 501 from peers:

Quote
received block 0000000046887292a76cd113a5fd6af38b17c9fb77e5936cd9856694030598f9
SetBestChain: new best=0000000046887292a76cd113a5fd6af38b17c9fb77e5936cd9856694030598f9  height=501  work=2156106482166  tx=510  date=2009-01-14 21:38:31
ProcessBlock: ACCEPTED
received block 000000003727134a4823b6c4c5245e961fff7816fc494742fb0abd353c94757b
SetBestChain: new best=000000003727134a4823b6c4c5245e961fff7816fc494742fb0abd353c94757b  height=502  work=2160401514999  tx=511  date=2009-01-14 21:46:15
ProcessBlock: ACCEPTED

So I quit again, deleted the blocks and chainstate folders and re-ran with the -reindex flag and that worked OK.


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: ghostshirt on February 12, 2013, 12:45:13 PM
I installed the new release on a relatively older Mac yesterday, It took about 8-9 hours of re-indexing blocks. It's been running smoothly for more than 24 hours. I tested a few transactions. Nothing unexpected to report.


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: wtfvanity on February 12, 2013, 04:14:08 PM
I went ahead and upgraded my public node to the .8 RC and in the past 36 hours, I have noticed that I have had an increase of almost 3x in the amount of bandwidth being used. it went from 2-3 GB outgoing per day to almost 10 GB outgoing per day. Is that because of the bloom filtering? Or just more people upgrading and testing out the RC?


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: zvs on February 12, 2013, 06:24:16 PM
I went ahead and upgraded my public node to the .8 RC and in the past 36 hours, I have noticed that I have had an increase of almost 3x in the amount of bandwidth being used. it went from 2-3 GB outgoing per day to almost 10 GB outgoing per day. Is that because of the bloom filtering? Or just more people upgrading and testing out the RC?

it seems to send blocks out to more people.  

when i look at my logs, i receive the same block a half dozen times usually.

using dstat, it will burst up to 100MB upstream sometimes, i guess from myself sending blocks to someone that has already received


like this:

2013-02-12 16:44:49 received block 00000000000004f958de0219d509e8ba156d9893f4903f9fa6c23edfed5ef565
2013-02-12 16:44:55 received block 00000000000004f958de0219d509e8ba156d9893f4903f9fa6c23edfed5ef565
2013-02-12 16:44:57 received block 00000000000004f958de0219d509e8ba156d9893f4903f9fa6c23edfed5ef565
2013-02-12 16:44:59 received block 00000000000004f958de0219d509e8ba156d9893f4903f9fa6c23edfed5ef565
2013-02-12 16:45:12 received block 00000000000004f958de0219d509e8ba156d9893f4903f9fa6c23edfed5ef565
2013-02-12 16:45:29 received block 00000000000000eb3c170ca260e1dffa2b1a91772dcae18a4df3446f377bd1b4
2013-02-12 16:46:09 received block 000000000000040681ba05ed54daa23667b2673d3ef6acbabd2dc36c95d66ab7
2013-02-12 16:48:44 received block 00000000000001f94f1644dfd3d056e5098d325b4b2bf4cb3c2690d7006c1a87
2013-02-12 17:01:41 received block 0000000000000379de774e67d9bdb305af68a956f891c548f0da19c538a79071
2013-02-12 17:01:44 received block 0000000000000379de774e67d9bdb305af68a956f891c548f0da19c538a79071
2013-02-12 17:01:47 received block 0000000000000379de774e67d9bdb305af68a956f891c548f0da19c538a79071
2013-02-12 17:01:56 received block 0000000000000379de774e67d9bdb305af68a956f891c548f0da19c538a79071
2013-02-12 17:02:08 received block 0000000000000379de774e67d9bdb305af68a956f891c548f0da19c538a79071
2013-02-12 17:10:07 received block 00000000000001cfaf40b52daf78d4cf9c0e93e427c242b90386ea6aae9d079b
2013-02-12 17:10:08 received block 00000000000001cfaf40b52daf78d4cf9c0e93e427c242b90386ea6aae9d079b
2013-02-12 17:10:11 received block 00000000000001cfaf40b52daf78d4cf9c0e93e427c242b90386ea6aae9d079b
2013-02-12 17:10:16 received block 00000000000001cfaf40b52daf78d4cf9c0e93e427c242b90386ea6aae9d079b
2013-02-12 17:10:31 received block 00000000000001cfaf40b52daf78d4cf9c0e93e427c242b90386ea6aae9d079b
2013-02-12 17:23:20 received block 0000000000000437a3ec102cf9050b7a26c0d986acf74cf9b3d6eff8ad7959de
2013-02-12 17:42:09 received block 00000000000004af3c30dac15e153941696c078a8e5c968bfe364de9a858e7c0
2013-02-12 17:57:25 received block 00000000000000576ea85bd5de07bdbc81a00549f6d20f8549b04a7597f5d64a
2013-02-12 17:57:26 received block 00000000000000576ea85bd5de07bdbc81a00549f6d20f8549b04a7597f5d64a
2013-02-12 17:57:33 received block 00000000000000576ea85bd5de07bdbc81a00549f6d20f8549b04a7597f5d64a
2013-02-12 17:57:39 received block 00000000000000576ea85bd5de07bdbc81a00549f6d20f8549b04a7597f5d64a
2013-02-12 17:58:02 received block 00000000000000576ea85bd5de07bdbc81a00549f6d20f8549b04a7597f5d64a
2013-02-12 18:00:04 received block 00000000000002bc74b17eb31d45dcac53ec1ba082c0bc7113e73ec32bae952f
2013-02-12 18:09:16 received block 000000000000015bd9e602c027f1eb2367ec7a8c8d4d70baa44cb27e86c6af81
2013-02-12 18:09:21 received block 000000000000015bd9e602c027f1eb2367ec7a8c8d4d70baa44cb27e86c6af81
2013-02-12 18:09:22 received block 000000000000015bd9e602c027f1eb2367ec7a8c8d4d70baa44cb27e86c6af81
2013-02-12 18:09:23 received block 000000000000015bd9e602c027f1eb2367ec7a8c8d4d70baa44cb27e86c6af81
2013-02-12 18:09:43 received block 000000000000015bd9e602c027f1eb2367ec7a8c8d4d70baa44cb27e86c6af81

and here is an example of network activity when a new block is found:

-net/total-
 recv  send
 104k  497k
 230k  708k
 143k  594k
  10k  271k
  10k  312k
7144B  223k
 204k  849k
  36k  470k
  98k  732k
  36k  364k
 115k  575k
 179k  785k
  73k  362k
 129k  570k
  15k  288k
 231k 1108k
 316k 1266k
  71k  435k
 116k  456k
 107k  564k
 149k 1058k
  24k  328k
8382B  202k
 129k  444k
 127k  539k
  15k  410k
  13k  459k
 262k  843k
 104k  451k
 286k  977k
 144k  724k
  21k  264k
  20k  385k
 164k  561k
  86k  524k
 277k 1068k
 229k  791k
 146k  522k
  15k  326k
 117k  439k
  22k  188k
 137k  531k
  45k  320k
 107k  575k
 222k  648k
  50k  283k
8960B  296k
  39k  376k
 725k   49M
 334k   68M
 176k   15M
 236k 3906k
 104k  671k
 151k 1093k
 131k  712k
  22k  549k
  26k  363k
 339k  989k
  47k  430k
 143k  603k
 131k  548k
  41k  281k
  29k  254k
 148k  709k
 182k  835k
 171k  620k
 130k  560k
 107k  529k
 124k  494k
  12k  226k
9887B  239k
6773B  177k
  96k  461k
 142k  529k
  12k  259k
 132k  527k
  10k  260k
 160k  546k
  44k 1148k
 114k  763k
  31k  381k
 147k  699k


this is with ~600 connections


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: mb300sd on February 12, 2013, 07:53:39 PM
I'm noticing the same issue with more bandwidth usage. Used to handle >200 connections fine but now its slowing down my connection.  Thought it was ISP issues until I checked.


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: dooglus on February 12, 2013, 08:04:19 PM
I just started the v0.8.0rc1-beta that I built myself from git and saw this splash screen:

http://i.imgur.com/OM3uWcN.png

Looks like the font is too big, or the message too long...


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: LightRider on February 13, 2013, 08:28:08 AM
Wow, I just noticed that I'm sending 1 mbit upstream as well. It would be nice if these (https://github.com/bitcoin/bitcoin/issues/273) issues (https://github.com/bitcoin/bitcoin/issues/1958) were addressed. Thanks again for all the work so far though.


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: dserrano5 on February 13, 2013, 09:11:18 AM
  • Installed 0.7.2 on a WinXP box (vmware virtual machine).
  • Downloaded some blocks, up to 135k or so.
  • Cleanly shut down bitcoin.
  • Installed 0.8.0rc1.
  • Bitcoin sucessfully reindexed the existing blocks, then started to download from that point.
  • Suspended the virtual machine.
  • Downloaded some more blocks from home, then suspended the VM again.
  • (related?) Installed Ares p2p software. Version 2.2.4 FWIW. (this was yesterday at home)
  • Today at work, downloaded more blocks.
  • While clicking around on Ares, I got a couple of dialog boxes about Visual C++ runtime <something> and mentioning bitcoin. However, I was able to click on the bitcoin window and bring it to top, giving it focus. I wasn't able to do that with Ares though (when I tried, one of the dialog boxes' title flashed, indicating me that I had to pay attention to it before using Ares again). So I happily disregarded the dialogs, only to see the bitcoin window disappear and the Ares one become unresponsive.
Code:
received block 00000000000000ad0e1ec1f69424abd586077ef7a7b953ecec8ebafa30a61ee2
ProcessBlock: ORPHAN BLOCK, prev=00000000000006c74bc0eaf942f5c93b53afd19c4b7d4b0ed35dcfe549ea824f
received block 00000000000006a58a50091c4feb7d8af30cb316ba6ecae8472efb29f125d9e5
ProcessBlock: ORPHAN BLOCK, prev=00000000000000ad0e1ec1f69424abd586077ef7a7b953ecec8ebafa30a61ee2
received block 000000000000051cfd88579acf5d0a5d20bd0232dc7ca174dd057e35ecf15187
ProcessBlock: ORPHAN BLOCK, prev=00000000000006a58a50091c4feb7d8af30cb316ba6ecae8472efb29f125d9e5
received block 000000000000006e3986d9f8510fc62cf8370678ca501ba24c228184cd268bef
ProcessBlock: ORPHAN BLOCK, prev=000000000000051cfd88579acf5d0a5d20bd0232dc7ca174dd057e35ecf15187


************************
EXCEPTION: St9bad_alloc      
std::bad_alloc      
C:\Archivos de programa\Bitcoin\bitcoin-qt.exe in ThreadMessageHandler()      



************************
EXCEPTION: St9bad_alloc      
std::bad_alloc      
C:\Archivos de programa\Bitcoin\bitcoin-qt.exe in ThreadSocketHandler()      

Flushed 12767 addresses to peers.dat  78ms

Sorry no timestamping (it should be on by default IMHO). Searching for "EXCEPTION" in the debug.log I find two additional bad_alloc messages. One of them appears 400 lines above this, so most probably it happened earlier today. The other one could be from yesterday, can't tell for sure. They could be related to the VM suspends.


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: dansmith on February 13, 2013, 10:28:44 AM

Upgraded a 32-bit system running 0.7.3 on Debian Squeeze.  The conversion of the old blocks seemed to slow down near the end, but left things running overnight and everything worked.


Pretty much a similar story here. On Ubuntu LTS running with -reindex.
A significant slowdown towards the last 4000 blocks. Took almost a day to reindex those 4000.

Another issue:
I had to reset my machine (cold restart) and now I get in terminal upon launching bitcoin-qt:
************************
EXCEPTION: 13leveldb_error       
Database I/O error       
bitcoin in Runaway exception       

I'm on ext4 with journaling.
Geez, I thought LevelDB on a journaling fs can't be corrupted by a reset.     


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: deepceleron on February 13, 2013, 10:46:15 AM
The checkpoint for this release is 216116, the current block is 220944. Full verification is done on block contents after the checkpoint, so it is expected to run slower on the last 4000 blocks.

I just finished an upgrade of a previously synchronized Bitcoin 0.7.2 on Win7, it took 70 minutes from startup to "up to date".

The Win32 client reports being built with Qt 4.8.3, the current release is v4.8.4, and has a good many fixes (http://qt.digia.com/Release-Notes/Release-Notes-Qt-484/).


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: Mike Hearn on February 13, 2013, 01:16:37 PM
Dan, I think there might be something wrong with your hard disk. Check /var/log/messages .... it shouldn't take a day to index 4000 blocks. Also if a hard disk is dying, getting very slow is one of the first symptoms.


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: dansmith on February 13, 2013, 02:24:32 PM
Here's the debug log's tail:

...
LoadBlockIndex(): transaction index disabled
LoadBlockIndex(): hashBestChain=0000000000000290e9c5ca7f9c4fb014c4f9f639dcffa6821c4616b7762509f7  height=220932 date=2013-02-13 09:02:08
init message: Verifying block database integrity...
Verifying last 288 blocks at level 3
LevelDB read failure: IO error: /home/default/.bitcoin/chainstate/029837.sst: No such file or directory


************************
EXCEPTION: 13leveldb_error       
Database I/O error       
bitcoin in Runaway exception       

Is there any way to recover from this error?
I double-checked that there's plenty of disk space left.


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: deepceleron on February 13, 2013, 02:40:30 PM
Here's the debug log's tail:

...
LoadBlockIndex(): transaction index disabled
LoadBlockIndex(): hashBestChain=0000000000000290e9c5ca7f9c4fb014c4f9f639dcffa6821c4616b7762509f7  height=220932 date=2013-02-13 09:02:08
init message: Verifying block database integrity...
Verifying last 288 blocks at level 3
LevelDB read failure: IO error: /home/default/.bitcoin/chainstate/029837.sst: No such file or directory


************************
EXCEPTION: 13leveldb_error       
Database I/O error       
bitcoin in Runaway exception       

Is there any way to recover from this error?
I double-checked that there's plenty of disk space left.

I have a feeling that the tech support answer for this will be "remove the index directories and reindex".

I would try fsck, and then look in lost+found for a 0-2MB file. Move it to the datadir as 029837.sst. See if Bitcoin doesn't stop crashing.


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: kgo on February 13, 2013, 04:20:22 PM
Working fine on a Macbook Pro OSX Snow Leopard.


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: saddambitcoin on February 13, 2013, 06:27:06 PM
upgraded on OSX 10.6.8, rescanning blocks took about 8 hrs but it seems to load faster now. connects to more peers than 0.7.2, before I would often be stuck at 8.


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: nibor on February 13, 2013, 11:45:43 PM
Ran with:

bitcoin-qt.exe -datadir=E:\bitcoin_database -dbcache=4000 -logtimestamps -txindex=1 -reindex=1

On i7 with 4 core+HT with 12gig ram on a nice new ssd on win7 64bit.

Took 30mins till started checking sigs. Then took another 30mins for last 4000ish blocks.


Used max of about 1.5-1.7gig.
Never read from disk for the chainstate - only wrote .logs
Only re-orged the blocks/index a few times.

Was entirely CPU bound. Initially on one thread then on all threads whilst checking sigs (ran at about 80% of the 8 virtual core busy which with hyperthreading is probably ideal).

Did a bit of checking in log file and are checking 1200 trans/second.


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: sharky112065 on February 14, 2013, 12:24:24 AM
Upgrade process went great for me. Took 23 minutes (Debian Squeeze, 32GB RAM, 8 Core CPU, and Hardware Raid 6). Shut it down, backed up data, removed blk* files, and restarted. No problems thus far. P2pool seems to like it just fine.


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: dooglus on February 14, 2013, 03:29:02 AM
Before the upgrade, I had:

Code:
  -rw-------  1 chris chris 2097307549 Aug  8  2012 blk0001.dat
  -rw-------  1 chris chris 2097177246 Dec  4 18:48 blk0002.dat
  -rw-------  1 chris chris 1420134352 Feb 12 11:52 blk0003.dat

After the upgrade, I have:

Code:
  -rw-------  2 chris chris 2097307549 Aug  8  2012 blocks/blk00000.dat
  -rw-------  2 chris chris 2097177246 Dec  4 18:48 blocks/blk00001.dat
  -rw-------  2 chris chris 1420134352 Feb 12 11:52 blocks/blk00002.dat
  -rw-------  1 chris chris   50331648 Feb 13 19:21 blocks/blk00003.dat

ie. each file's number went down by one, and a new file was started even though the previous one (was #3, now #2) didn't get anywhere near to 2GB.

Is that normal?


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: foo on February 14, 2013, 05:31:18 AM
ie. each file's number went down by one, and a new file was started even though the previous one (was #3, now #2) didn't get anywhere near to 2GB.

Is that normal?

Yes. The new code creates 128 MB block files, but it will keep the old ones as is.


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: tHash on February 14, 2013, 11:17:35 PM
Around block 188,000, bitcoin threw an error that it "could not write block" and exited.   Continuing to download the blockchain without issue after a restart.

This is a fresh install on Windows 7 x64.


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: grue on February 14, 2013, 11:32:39 PM
Fully syncs up without a problem on Windows 7 x64. Haven't done any transactions on it though.


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: fornit on February 15, 2013, 02:55:20 AM
using win7 64bit.

during sync the client crashed twice claiming the db is corrupted. first time restarting the client helped, second time i had to reboot. now its running smooth.


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: ArticMine on February 15, 2013, 02:58:28 AM

...

Improvements
------------

Mac and Windows binaries are signed with certificates owned by the Bitcoin
Foundation, to be compatible with the new security features in OSX 10.8 and
Windows 8.

...


What happens if these certificates are revoked? Could this lead to a Treacherous Computing, http://www.gnu.org/philosophy/can-you-trust.html (http://www.gnu.org/philosophy/can-you-trust.html), attack against Bitcoin?


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: grue on February 15, 2013, 03:14:16 AM
Mac and Windows binaries are signed with certificates owned by the Bitcoin
Foundation, to be compatible with the new security features in OSX 10.8 and
Windows 8.
are these signatures only visible in windows 8? because i'm looking at the binary and it does not have a digital signature.


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: Gavin Andresen on February 15, 2013, 04:59:54 AM
The windows setup.exe is signed, as is the Mac .app bundle. The executables inside them are not signed (I can't think of a good reason to sign them, it would not increase download security at all).

You can also still check the download packages using the SHASUMS.asc file, which is signed with my gpg key.

And if you are running Linux or Windows you could check all of the files in the installer against other core developer's keys.

If the code signing certificate was revoked then we would go back to just using gpg keys. The code signing certificate is great because Windows and OSX know how to check it automatically when the download happens.


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: ArticMine on February 15, 2013, 05:41:34 AM
The windows setup.exe is signed, as is the Mac .app bundle. The executables inside them are not signed (I can't think of a good reason to sign them, it would not increase download security at all).

You can also still check the download packages using the SHASUMS.asc file, which is signed with my gpg key.

And if you are running Linux or Windows you could check all of the files in the installer against other core developer's keys.

If the code signing certificate was revoked then we would go back to just using gpg keys. The code signing certificate is great because Windows and OSX know how to check it automatically when the download happens.

Thank you very much for the clarification. My concern was actually would the .exe would continue to run under Windows 8 if the certificate was revoked after installation? I believe that under Windows XP/Vista/7 the .exe would continue to run, but I am not clear under Windows 8.  The possible danger here is Microsoft being able to shut-down Bitcoin-Qt / bitcoind after installation by revoking the certificate hence the reference to a Treacherous Computing attack.

I did run into a somewhat similar situation back in 2005 where an expired certificate on APC PowerChute not only prevented the software from installing but also caused significant damage to the Windows registry. I spent the better part of a Sunday afternoon fixing the Windows registry in order to bring the server back to health  The OS was Windows Server 2003. Granted this was an installation faliure but given the direction that Microsoft is taking with Windows 8 RT, I would be rather safe than sorry when it comes to Windows 8 on Intel/AMD.


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: Mike Hearn on February 15, 2013, 09:21:08 AM
I think we should be signing the Windows binaries as well. Binary signatures are not only used by browsers and the OS when running setup programs. Anti-virus scanners also use them to build binary reputations and avoid false positives. Bitcoin does things that can look a tiny bit malware like (connecting to P2P networks and exchanging lots of random-looking data), so signing the executables can't hurt.

I don't think Microsoft doing an Apple and trying to ban unsigned code is going to ever happen, at least not on regular desktop Windows. There are far, far, far too many legacy/in-house apps out there that are business critical and yet never been signed, never will be signed.


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: grue on February 15, 2013, 02:45:04 PM
The windows setup.exe is signed, as is the Mac .app bundle. The executables inside them are not signed (I can't think of a good reason to sign them, it would not increase download security at all).

You can also still check the download packages using the SHASUMS.asc file, which is signed with my gpg key.

And if you are running Linux or Windows you could check all of the files in the installer against other core developer's keys.

If the code signing certificate was revoked then we would go back to just using gpg keys. The code signing certificate is great because Windows and OSX know how to check it automatically when the download happens.
I don't use the setup because I want to avoid going through tedious steps with the installer. most software makers sign both their installer AND their binary, so I don't see why bitcoin should be any different.


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: dansmith on February 15, 2013, 06:55:08 PM
I again received:

Quote
************************
EXCEPTION: 13leveldb_error       
Database I/O error       
bitcoin in Runaway exception 

after I made a hard reset of my machine.
Harddisk is not to blame, since last time I got this error, I fsck'ed my ext4 root partition from a LiveCD and it came out clean.

(Good job I backed-up index,chainstate, and database dirs, so I restored them without having to reindex like I did last time).


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: zvs on February 15, 2013, 08:04:08 PM
I again received:

Quote
************************
EXCEPTION: 13leveldb_error       
Database I/O error       
bitcoin in Runaway exception 

after I made a hard reset of my machine.
Harddisk is not to blame, since last time I got this error, I fsck'ed my ext4 root partition from a LiveCD and it came out clean.

(Good job I backed-up index,chainstate, and database dirs, so I restored them without having to reindex like I did last time).

i actually had to do this reindex process about half a dozen times... but i'd stop it every so often to make a backup & it finally finished.

it seemed to have some random chance to get into a loop where it gets orphan blocks over and over


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: keystroke on February 16, 2013, 02:29:16 AM
Starts up fine on Windows 8 and seems ok. Doing the block reindexing now. Nice work everyone!

Quick question: Why no more detach database on shutdown?

Edit: Took just under 2 hours on a i5-560m.
Client starts and stops super fast.

Is the old database removed or can I delete something to save space?


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: dooglus on February 16, 2013, 08:13:58 AM
Is the old database removed or can I delete something to save space?

I think the only file you can delete to save space is blkindex.dat in the same folder as wallet.dat.  That frees up around 1.8GB.

You could also delete the blk000*.dat files from the same folder, but that shouldn't free up any space, because there are links to the same files in the blocks/ subfolder.  Unless your filesystem doesn't support hardlinks, in which case deleting the blk000*.dat files in the same folder as wallet.dat will free up 5.5GB or so.


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: Pieter Wuille on February 16, 2013, 12:21:11 PM
Quick question: Why no more detach database on shutdown?

Because the block database(s) are LevelDB now. The BDB databases use a "database environment" (a directory shared by all databases) for journalling and consistency checks. To support moving the database files around, the files were "detached" from the environment at shutdown. Originally, this was always done. Since 0.6.1 this was made optional for the block chain databases (wallets were still always detached at shutdown), and -detachdb was introduced to re-enable it.

LevelDB databases use an entire directory by themselves, and are independent. So there is no environment to detach from anymore. The wallet in 0.8 is still BDB, and is still always detached.

Quote
Is the old database removed or can I delete something to save space?

No, it's not. But if you don't intend to go back to 0.7.x or earlier, you can delete blkindex.dat, blk0001.dat and blk0002.dat from the datadir (don't delete anything inside the blocks/ subdirectory though).


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: Pieter Wuille on February 16, 2013, 12:32:17 PM
i actually had to do this reindex process about half a dozen times... but i'd stop it every so often to make a backup & it finally finished.

it seemed to have some random chance to get into a loop where it gets orphan blocks over and over

If you see the reindexing finding orphans over and over, it usually means one block couldn't be found, but laters blocks can be. Since one block in their ancestry can't be connected, it is reported as an orphan and skipped. It's not an infinite loop though, it will just go over all block files. We've had some bugs in pre-releases that could cause this, but for someone first trying 0.8.0rc1, my guess is that you had a few blocks on disk that actually were corrupted.

How did you fix it? If my assumption is correct, the only solution would be waiting a few minutes for it to skip all blocks you already have, and then waiting for it to start downloading from before the invalid block.


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: hazek on February 16, 2013, 09:45:27 PM
Gavin & co. Can't you guys speed things up a bit? Right now we're getting a lot of good quality publicity and I think it would be a real shame to see all these new potential users get to bitcoin.org and download 7.2 which is very likely going severely disappoint them..

If there aren't any bugs, and there appear to not be any reports of bugs, can you move things a long a bit faster? Or is there something that still needs to be done?


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: dooglus on February 16, 2013, 11:08:46 PM
Gavin & co. Can't you guys speed things up a bit? Right now we're getting a lot of good quality publicity and I think it would be a real shame to see all these new potential users get to bitcoin.org and download 7.2 which is very likely going severely disappoint them..

If there aren't any bugs, and there appear to not be any reports of bugs, can you move things a long a bit faster? Or is there something that still needs to be done?

I'd rather have a stable release than an early release one if that's the choice.


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: twobitcoins on February 17, 2013, 05:12:24 AM
Am testing it now, completely fresh install on a netbook, Windows 8 64-bit, AMD E-350, 6 GB of RAM, 5400 rpm hard drive, using -dbcache=2000.

Will report back once it's done.

Bummer, midway through downloading the blockchain, with -dbcache=2000, it kept using up RAM all the way to 4GB and then crashed.

On restart, it sits at the wallet picture with "xporting blocks from block databas" (text cut off on left and right hand side).  I can see from the Task Manager that it's doing something, pegged CPU at 50%, and increasing RAM. I'll let it sit and see where it goes.

Edit: It finally came out of the splash screen and CPU usage dropped to normal, but RAM is still increasing. I've got it set to -dbcache=1000 this time, so we'll see if it sticks to that.

I closed the instance running -dbcache=1000 to see how it would handle it, and it closed no problem. I've restarted it with -dbcache=1500, and have let it run for a while. Memory use is now up to 1,660,316 KB, so, similar to my results using -dbcache=2000 where it went up to 4GB, it doesn't seem to be minding the value chosen in this setting as a limit.

I see similar issues with memory usage.  I built bitcoin-qt v0.8.0rc1 on Windows and tested it by downloading blocks from scratch, clearing the data directory each time.

On the first attempt, I ran with "dbcache=1000" and "txindex=1" in bitcoin.conf (though I'm not sure the txindex setting had any effect, because I saw "LoadBlockIndex(): transaction index disabled" in debug.log).  It eventually crashed, and there were many bad_alloc exceptions in debug.log.

On the second attempt, I did not use any special options.  Memory usage gradually increased to 2200MB while downloading blocks, and then decreased to 1650MB once downloading was finished.  Perhaps the decrease was a result of freeing the blocks stored in mapOrphanBlocks, which I believe contained several thousand blocks that had been downloaded in reverse.

It seems there is a memory leak during initial block download.  If this will affect all new users, it seems rather important to fix.


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: zvs on February 17, 2013, 10:55:13 AM
i actually had to do this reindex process about half a dozen times... but i'd stop it every so often to make a backup & it finally finished.

it seemed to have some random chance to get into a loop where it gets orphan blocks over and over

If you see the reindexing finding orphans over and over, it usually means one block couldn't be found, but laters blocks can be. Since one block in their ancestry can't be connected, it is reported as an orphan and skipped. It's not an infinite loop though, it will just go over all block files. We've had some bugs in pre-releases that could cause this, but for someone first trying 0.8.0rc1, my guess is that you had a few blocks on disk that actually were corrupted.

How did you fix it? If my assumption is correct, the only solution would be waiting a few minutes for it to skip all blocks you already have, and then waiting for it to start downloading from before the invalid block.

I didn't fix it per se, I just would abort the reindex process every so often and copy the files to a backup directory (hmm, starting around block 210k).  Every 2000 or 3000 blocks, I'd stop the process, copy everything over, and start it again.  Sometimes it'd get into the orphan block loop and sometimes it wouldn't...  eventually it was able to complete the whole process (w/o me changing anything in regard to what it was using to reindex)


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: phatsphere on February 17, 2013, 09:16:45 PM
and just one other thing, there are no menus in linux with ubuntu's unity. does anyone test this with this (possibly most popular) window manager in linux? there is also no application icon either.


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: grue on February 18, 2013, 01:14:13 AM
bad_alloc = not enough ram


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: dust on February 18, 2013, 02:16:11 AM
I just synced the entire blockchain in about 50 minutes on a new Mac mini with an i7-3720QM (fast) and the stock 5400 rpm hd (slow) using -dbcache=4096 -par=8 and -connect to a server on the local network.  bitcoind is running in an Ubuntu VM and there have been no crashes so far.  I was also compiling and syncing a bunch of altcoins in the background.


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: dooglus on February 18, 2013, 04:24:45 AM
and just one other thing, there are no menus in linux with ubuntu's unity. does anyone test this with this (possibly most popular) window manager in linux? there is also no application icon either.

I tested in Ubuntu but not using Unity.  I'll take a look now at how it works in Unity.

Edit: I don't see the menus in Unity either, but do when using XFCE4.  In Unity each application's menu bar is supposed to appear at the very top of the screen when you move the mouse up there.  Here's an example where it works, with a terminal window:

http://i.imgur.com/9tTvNYW.png

but with bitcoin 0.8rc1 the menu doesn't appear when I mouse up to the top of the screen:

http://i.imgur.com/tSWD44H.png

In bitcoin 0.7 the menu does appear as it should:

http://i.imgur.com/224z8S6.png


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: caveden on February 18, 2013, 07:19:31 AM
Gavin & co. Can't you guys speed things up a bit? Right now we're getting a lot of good quality publicity and I think it would be a real shame to see all these new potential users get to bitcoin.org and download 7.2 which is very likely going severely disappoint them..

If there aren't any bugs, and there appear to not be any reports of bugs, can you move things a long a bit faster? Or is there something that still needs to be done?

I'd rather have a stable release than an early release one if that's the choice.

I agree, but I also agree with hazek in the sense that it's not cool to turn off new comers. So, maybe we should not direct them immediately to bitcoin-qt on bitcoin.org... maybe we should make sure they see a list of available clients before downloading any?


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: novusordo on February 19, 2013, 12:12:33 AM
Gavin & co. Can't you guys speed things up a bit? Right now we're getting a lot of good quality publicity and I think it would be a real shame to see all these new potential users get to bitcoin.org and download 7.2 which is very likely going severely disappoint them..

If there aren't any bugs, and there appear to not be any reports of bugs, can you move things a long a bit faster? Or is there something that still needs to be done?

I'd rather have a stable release than an early release one if that's the choice.

I agree, but I also agree with hazek in the sense that it's not cool to turn off new comers. So, maybe we should not direct them immediately to bitcoin-qt on bitcoin.org... maybe we should make sure they see a list of available clients before downloading any?

It'd be nice if the download section on the right of bitcoin.org had one link to the clients (http://bitcoin.org/clients.html) page, and nothing else. New users generally are going to click on the first download button they see, but if they are presented with multiple options they will be more inclined to either experiment a little or research the different types of clients.


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: Mike Hearn on February 19, 2013, 10:06:32 AM
Yeah, we want to do that too. Actually Gavin asked me to do it last night as I'm working on the website at the moment.

It needs a bit more than just a link to the Clients page, IMHO, but we can thrash out the details on a pull request.


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: fornit on February 19, 2013, 05:57:30 PM
I again received:

Quote
************************
EXCEPTION: 13leveldb_error       
Database I/O error       
bitcoin in Runaway exception 

after I made a hard reset of my machine.
Harddisk is not to blame, since last time I got this error, I fsck'ed my ext4 root partition from a LiveCD and it came out clean.

(Good job I backed-up index,chainstate, and database dirs, so I restored them without having to reindex like I did last time).

i actually had to do this reindex process about half a dozen times... but i'd stop it every so often to make a backup & it finally finished.

it seemed to have some random chance to get into a loop where it gets orphan blocks over and over

i get those too, and its a completely fresh windows 7 and bitcoin 0.8 install.


Title: Re: Bitcoin-Qt / bitcoind version 0.8.0 release candidate 1
Post by: mmortal03 on February 19, 2013, 07:49:34 PM
Am testing it now, completely fresh install on a netbook, Windows 8 64-bit, AMD E-350, 6 GB of RAM, 5400 rpm hard drive, using -dbcache=2000.

Will report back once it's done.

Bummer, midway through downloading the blockchain, with -dbcache=2000, it kept using up RAM all the way to 4GB and then crashed.

On restart, it sits at the wallet picture with "xporting blocks from block databas" (text cut off on left and right hand side).  I can see from the Task Manager that it's doing something, pegged CPU at 50%, and increasing RAM. I'll let it sit and see where it goes.

Edit: It finally came out of the splash screen and CPU usage dropped to normal, but RAM is still increasing. I've got it set to -dbcache=1000 this time, so we'll see if it sticks to that.

I closed the instance running -dbcache=1000 to see how it would handle it, and it closed no problem. I've restarted it with -dbcache=1500, and have let it run for a while. Memory use is now up to 1,660,316 KB, so, similar to my results using -dbcache=2000 where it went up to 4GB, it doesn't seem to be minding the value chosen in this setting as a limit.

I see similar issues with memory usage.  I built bitcoin-qt v0.8.0rc1 on Windows and tested it by downloading blocks from scratch, clearing the data directory each time.

On the first attempt, I ran with "dbcache=1000" and "txindex=1" in bitcoin.conf (though I'm not sure the txindex setting had any effect, because I saw "LoadBlockIndex(): transaction index disabled" in debug.log).  It eventually crashed, and there were many bad_alloc exceptions in debug.log.

On the second attempt, I did not use any special options.  Memory usage gradually increased to 2200MB while downloading blocks, and then decreased to 1650MB once downloading was finished.  Perhaps the decrease was a result of freeing the blocks stored in mapOrphanBlocks, which I believe contained several thousand blocks that had been downloaded in reverse.

It seems there is a memory leak during initial block download.  If this will affect all new users, it seems rather important to fix.


I'd run it from scratch again myself without any special options, but something about having all the connections from Bitcoin over time causes the router I share with others to slow to a halt, and I don't have access to it at the moment to reset it when this happens.

Which version of Windows are you using?