Bitcoin-Qt, bitcoind testing requested! You can help.

[BkkCoins]

I've been  running the new bitcoind and waiting for it to catch up. It seems to have aborted several times during this process as each time I come back later it has exited and not completed the blockchain update.

I looked thru the debug.log file for some messages but only thing I've seen is what appears to be a normal exit. However, just now I checked and saw some output from the getinfo rpc command. Checking the debug.log I see the following.

Code:

ProcessBlock: ACCEPTED
received block 00000000000002b65a1099bad8ac9b13fb8ef6b4ee37d8060e5952bd38bc6699
SetBestChain: new best=00000000000002b65a1099bad8ac9b13fb8ef6b4ee37d8060e5952bd38bc6699  height=208886  work=612743795138239981020  tx=9107953  date=11/21/12 10:39:23
ProcessBlock: ACCEPTED
stored orphan tx d93145a3b5 (mapsz 4114)
stored orphan tx 6f4a89cce4 (mapsz 4115)
received block 00000000000000fdc90808f26bed44dc04de7358216c64a44aef133678984291
ERROR: ProcessBlock() : already have block 208730 00000000000000fdc90808f26bed44dc04de7358216c64a44aef133678984291
received block 00000000000003fde311970bb97326fcff81e7495105ab92a9cb82695e0582a4

Nor sure why it jumped from block 208886 to 208730 without messages between. But noticed the error. This was also shown in getinfo rpc call as,

Code:

{
    "version" : 79900,
    "protocolversion" : 60002,
    "walletversion" : 60000,
    "balance" : 0.00000000,
    "blocks" : 208843,
    "connections" : 16,
    "proxy" : "",
    "difficulty" : 3368767.14053294,
    "testnet" : false,
    "keypoololdest" : 1354352653,
    "keypoolsize" : 101,
    "paytxfee" : 0.00000000,
    "errors" : "EXCEPTION: St9bad_alloc       \nstd::bad_alloc       \nbitcoin in ProcessMessages()       \n"
}

Further attempts to check progress with getinfo are very slow or hung. Tried to stop and start again but seems to have not exited as unable to bind to 0.0.0.0 at start. Retry after a while works. Another getinfo indicates block 209579 - so it seems to have continued on.

[BkkCoins]

Trying to build bitcoin on an Amazon EC2 instance. I'm using git cloned source.

It compiles fine until this,

Code:

...
g++ -c -O2 -pthread -Wall -Wextra -Wformat -Wformat-security -Wno-unused-parameter -g -DBOOST_SPIRIT_THREADSAFE -I/usr/src/bitcoin/src -I/usr/src/bitcoin/src/obj -DUSE_IPV6=1 -I/usr/src/bitcoin/src/leveldb/include -I/usr/src/bitcoin/src/leveldb/helpers -DHAVE_BUILD_INFO -fno-stack-protector -fstack-protector-all -Wstack-protector -D_FORTIFY_SOURCE=2  -MMD -MF obj/bitcoinrpc.d -o obj/bitcoinrpc.o bitcoinrpc.cpp
g++: internal compiler error: Killed (program cc1plus)
Please submit a full bug report,
with preprocessed source if appropriate.
See <file:///usr/share/doc/gcc-4.6/README.Bugs> for instructions.
make: *** [obj/bitcoinrpc.o] Error 4

Cannot continue now after several attempts. Seems to be tool problem as this did build on my other vps also 12.04.

gcc 4.6.3

My tools installed like this:
sudo apt-get install git htop build-essential libssl-dev libboost-all-dev libdb5.1++-dev

Anyone have ideas how to get around this? thx.

[Luke-Jr]

Trying to build bitcoin on an Amazon EC2 instance. I'm using git cloned source.

It compiles fine until this,

Code:

...
g++ -c -O2 -pthread -Wall -Wextra -Wformat -Wformat-security -Wno-unused-parameter -g -DBOOST_SPIRIT_THREADSAFE -I/usr/src/bitcoin/src -I/usr/src/bitcoin/src/obj -DUSE_IPV6=1 -I/usr/src/bitcoin/src/leveldb/include -I/usr/src/bitcoin/src/leveldb/helpers -DHAVE_BUILD_INFO -fno-stack-protector -fstack-protector-all -Wstack-protector -D_FORTIFY_SOURCE=2  -MMD -MF obj/bitcoinrpc.d -o obj/bitcoinrpc.o bitcoinrpc.cpp
g++: internal compiler error: Killed (program cc1plus)
Please submit a full bug report,
with preprocessed source if appropriate.
See <file:///usr/share/doc/gcc-4.6/README.Bugs> for instructions.
make: *** [obj/bitcoinrpc.o] Error 4

Cannot continue now after several attempts. Seems to be tool problem as this did build on my other vps also 12.04.

gcc 4.6.3

My tools installed like this:
sudo apt-get install git htop build-essential libssl-dev libboost-all-dev libdb5.1++-dev

Anyone have ideas how to get around this? thx.

Likely it doesn't have enough memory.

[BkkCoins]

Trying to build bitcoin on an Amazon EC2 instance. I'm using git cloned source.

It compiles fine until this,

Code:

...
g++ -c -O2 -pthread -Wall -Wextra -Wformat -Wformat-security -Wno-unused-parameter -g -DBOOST_SPIRIT_THREADSAFE -I/usr/src/bitcoin/src -I/usr/src/bitcoin/src/obj -DUSE_IPV6=1 -I/usr/src/bitcoin/src/leveldb/include -I/usr/src/bitcoin/src/leveldb/helpers -DHAVE_BUILD_INFO -fno-stack-protector -fstack-protector-all -Wstack-protector -D_FORTIFY_SOURCE=2  -MMD -MF obj/bitcoinrpc.d -o obj/bitcoinrpc.o bitcoinrpc.cpp
g++: internal compiler error: Killed (program cc1plus)
Please submit a full bug report,
with preprocessed source if appropriate.
See <file:///usr/share/doc/gcc-4.6/README.Bugs> for instructions.
make: *** [obj/bitcoinrpc.o] Error 4

Cannot continue now after several attempts. Seems to be tool problem as this did build on my other vps also 12.04.

gcc 4.6.3

My tools installed like this:
sudo apt-get install git htop build-essential libssl-dev libboost-all-dev libdb5.1++-dev

Anyone have ideas how to get around this? thx.

Likely it doesn't have enough memory.

Hmmm. If that's true then it's totally annoying as I built it yesterday on another system with less memory.
Yesterday on 32 bit Ubuntu, 512 MB, VPS and it works. Today on new EC2 instance I'm trying 64 bit, as that's what's recommended now, and it has 612 MB (t1.micro). I copied over blockchain data but cannot copy the program (due to difference 32 -64 bit). Blast it!

I'll have to figure out how to freeze the instance and restart it as a t1.small instance for compiling.

[Luke-Jr]

Trying to build bitcoin on an Amazon EC2 instance. I'm using git cloned source.

It compiles fine until this,

Code:

...
g++ -c -O2 -pthread -Wall -Wextra -Wformat -Wformat-security -Wno-unused-parameter -g -DBOOST_SPIRIT_THREADSAFE -I/usr/src/bitcoin/src -I/usr/src/bitcoin/src/obj -DUSE_IPV6=1 -I/usr/src/bitcoin/src/leveldb/include -I/usr/src/bitcoin/src/leveldb/helpers -DHAVE_BUILD_INFO -fno-stack-protector -fstack-protector-all -Wstack-protector -D_FORTIFY_SOURCE=2  -MMD -MF obj/bitcoinrpc.d -o obj/bitcoinrpc.o bitcoinrpc.cpp
g++: internal compiler error: Killed (program cc1plus)
Please submit a full bug report,
with preprocessed source if appropriate.
See <file:///usr/share/doc/gcc-4.6/README.Bugs> for instructions.
make: *** [obj/bitcoinrpc.o] Error 4

Cannot continue now after several attempts. Seems to be tool problem as this did build on my other vps also 12.04.

gcc 4.6.3

My tools installed like this:
sudo apt-get install git htop build-essential libssl-dev libboost-all-dev libdb5.1++-dev

Anyone have ideas how to get around this? thx.

Likely it doesn't have enough memory.

Hmmm. If that's true then it's totally annoying as I built it yesterday on another system with less memory.
Yesterday on 32 bit Ubuntu, 512 MB, VPS and it works. Today on new EC2 instance I'm trying 64 bit, as that's what's recommended now, and it has 612 MB (t1.micro). I copied over blockchain data but cannot copy the program (due to difference 32 -64 bit). Blast it!

I'll have to figure out how to freeze the instance and restart it as a t1.small instance for compiling.

64-bit generally uses up to 2 times as much memory for the same data structures.

[BkkCoins]

Trying to build bitcoin on an Amazon EC2 instance. I'm using git cloned source.

It compiles fine until this,

Code:

...
g++ -c -O2 -pthread -Wall -Wextra -Wformat -Wformat-security -Wno-unused-parameter -g -DBOOST_SPIRIT_THREADSAFE -I/usr/src/bitcoin/src -I/usr/src/bitcoin/src/obj -DUSE_IPV6=1 -I/usr/src/bitcoin/src/leveldb/include -I/usr/src/bitcoin/src/leveldb/helpers -DHAVE_BUILD_INFO -fno-stack-protector -fstack-protector-all -Wstack-protector -D_FORTIFY_SOURCE=2  -MMD -MF obj/bitcoinrpc.d -o obj/bitcoinrpc.o bitcoinrpc.cpp
g++: internal compiler error: Killed (program cc1plus)
Please submit a full bug report,
with preprocessed source if appropriate.
See <file:///usr/share/doc/gcc-4.6/README.Bugs> for instructions.
make: *** [obj/bitcoinrpc.o] Error 4

Cannot continue now after several attempts. Seems to be tool problem as this did build on my other vps also 12.04.

gcc 4.6.3

My tools installed like this:
sudo apt-get install git htop build-essential libssl-dev libboost-all-dev libdb5.1++-dev

Anyone have ideas how to get around this? thx.

Likely it doesn't have enough memory.

Hmmm. If that's true then it's totally annoying as I built it yesterday on another system with less memory.
Yesterday on 32 bit Ubuntu, 512 MB, VPS and it works. Today on new EC2 instance I'm trying 64 bit, as that's what's recommended now, and it has 612 MB (t1.micro). I copied over blockchain data but cannot copy the program (due to difference 32 -64 bit). Blast it!

I'll have to figure out how to freeze the instance and restart it as a t1.small instance for compiling.

64-bit generally uses up to 2 times as much memory for the same data structures.

Ok. Didn't know that as have always worked on 32 bit here. I think I'll revert to a 32 bit instance then as even on my 512 MB VPS running the server has been tight for memory. It likely won't run well as 64 bit.

Thx.

[jgarzik]

Trying to build bitcoin on an Amazon EC2 instance. I'm using git cloned source.

It compiles fine until this,
[...]
Cannot continue now after several attempts. Seems to be tool problem as this did build on my other vps also 12.04.

My tools installed like this:
sudo apt-get install git htop build-essential libssl-dev libboost-all-dev libdb5.1++-dev

Anyone have ideas how to get around this? thx.

As Luke guessed, this is the compiler running out of memory.

You can always compile elsewhere, and copy the 'bitcoind' binary to the VPS for running.

There is no requirement that you compile on the same host that will run bitcoind.

[BkkCoins]

Trying to build bitcoin on an Amazon EC2 instance. I'm using git cloned source.

It compiles fine until this,
[...]
Cannot continue now after several attempts. Seems to be tool problem as this did build on my other vps also 12.04.

My tools installed like this:
sudo apt-get install git htop build-essential libssl-dev libboost-all-dev libdb5.1++-dev

Anyone have ideas how to get around this? thx.

As Luke guessed, this is the compiler running out of memory.

You can always compile elsewhere, and copy the 'bitcoind' binary to the VPS for running.

There is no requirement that you compile on the same host that will run bitcoind.

Thx. I realized after that I had all the blockchain data on a separate volume. So I detached it and started another instance running m1.small with 1.7G ram instead.

My plan was to post a tutorial on creating an Electrum AMI so that people could start up new spot instance nodes with just a few clicks. But I started to doubt the economy of that as EC2 is quite expensive compared to low end VPS hosts. The micro instances are quite pathetic in cpu allocation so once you move to a small instance (no speed demon either), and include storage fees, the cost starts to become unattractive.

This was just experimental as I already have a VPS running Bitcoin which only costs $10/qtr.

[flatfly]

This was just experimental as I already have a VPS running Bitcoin which only costs $10/qtr.

Interesting! Would you mind sharing more info on your VPS provider?

[BkkCoins]

This was just experimental as I already have a VPS running Bitcoin which only costs $10/qtr.

Interesting! Would you mind sharing more info on your VPS provider?

Sure. I found it by browsing thru lowendbox.com. They have a nice listing of cheap vps providers.
The one I'm using is cloud3k.com but there were a few at the time that were competitive with similar prices. I would upgrade to 1G RAM but it actually more than doubles the cost.

The only issue I've had was when I tried to register a free subdomain at freedns - they rejected my IP as on a black list. That's why I went with dot.tk instead. Not black listed there. I didn't want to hassle with contacting support and trying to change it. Performance has been fine and I've never noticed any down time.

I'm not sure if I trust them to the extent I would Amazon EC2 and I'd be leary of leaving any hot wallet there, but other than that I haven't had other problems. I run my "real" web site on EC2. The performance is poor and price higher but I believe it to be more robust.

edit: I should note that I get exceptions/crashes periodically from bitcoind - probably related to not enough memory since it appears to be a memory alloc exception. So 512MB is not really enough to run both these servers. I don't know why so much has to be kept in memory but from a practical pov it kind of means that to be stable users need to spend more money for a beefier server. If the devs are serious about getting more nodes running maybe working on reducing memory footprint would help make it less costly to run nodes.

[flatfly]

This was just experimental as I already have a VPS running Bitcoin which only costs $10/qtr.

Interesting! Would you mind sharing more info on your VPS provider?

Sure. I found it by browsing thru lowendbox.com. They have a nice listing of cheap vps providers.
The one I'm using is cloud3k.com but there were a few at the time that were competitive with similar prices. I would upgrade to 1G RAM but it actually more than doubles the cost.

The only issue I've had was when I tried to register a free subdomain at freedns - they rejected my IP as on a black list. That's why I went with dot.tk instead. Not black listed there. I didn't want to hassle with contacting support and trying to change it. Performance has been fine and I've never noticed any down time.

I'm not sure if I trust them to the extent I would Amazon EC2 and I'd be leary of leaving any hot wallet there, but other than that I haven't had other problems. I run my "real" web site on EC2. The performance is poor and price higher but I believe it to be more robust.

edit: I should note that I get exceptions/crashes periodically from bitcoind - probably related to not enough memory since it appears to be a memory alloc exception. So 512MB is not really enough to run both these servers. I don't know why so much has to be kept in memory but from a practical pov it kind of means that to be stable users need to spend more money for a beefier server. If the devs are serious about getting more nodes running maybe working on reducing memory footprint would help make it less costly to run nodes.

Thanks for the detailed answer!

[BkkCoins]

I'm probably going to scrap the 512MB VPS now. I get shutdowns, apparently mem allocs, pretty consistently after an hour or two. So I just think this isn't enough memory. I'm back working on another EC2 build, doing the compiling and initial block download on a c1.medium (high cpu) instance. Then I'll change to normal operation on a m1.small instance for economy. My estimate is that this will cost about $6.44/mo. ($5.04 cpu + $1.40 storage). Both these instances have 1.7G memory but the small instance has whimpy cpu speed, though I don't see that being a problem unless it gets quite busy.

This is using spot instances so there will be occasional brief outages when the bid prices go crazy. A spot instance can be made "persistent" so that it restarts when bids come back down and instances are available. Anyway, I'm keeping notes so I can post a tutorial on creating an Electrum AMI for this type of use.