Bitcoin Forum

Bitcoin => Bitcoin Technical Support => Topic started by: crazy_rabbit on March 10, 2013, 07:23:30 PM



Title: How large an Amazon EC2 instance for Bitcoin?
Post by: crazy_rabbit on March 10, 2013, 07:23:30 PM
I'm running a bitcoin service on amazon Ec2, and I need to run a full version of bitcoin- block chain included. How much disk space will I need to keep on hand to keep bitcoind from crashing? (my current problem).


Title: Re: How large an Amazon EC2 instance for Bitcoin?
Post by: 2112 on March 10, 2013, 10:10:52 PM
I'm running a bitcoin service on amazon Ec2, and I need to run a full version of bitcoin- block chain included. How much disk space will I need to keep on hand to keep bitcoind from crashing? (my current problem).
The smallest has 160GB of the local storage. That is more than enough for couple of years. I/O to the local storage is free of charge. Just don't pull a bitomat.pl and don't store the backup copy of the wallet on the same ephemeral storage as the working blockchain.


Title: Re: How large an Amazon EC2 instance for Bitcoin?
Post by: Sukrim on March 11, 2013, 12:33:30 AM
Protipp: Use a BIP32 deterministic wallet to create addresses on the fly on the server but keep the private keys offline in your wallet.


Title: Re: How large an Amazon EC2 instance for Bitcoin?
Post by: crazy_rabbit on March 11, 2013, 07:49:39 PM
I'm running a bitcoin service on amazon Ec2, and I need to run a full version of bitcoin- block chain included. How much disk space will I need to keep on hand to keep bitcoind from crashing? (my current problem).
The smallest has 160GB of the local storage. That is more than enough for couple of years. I/O to the local storage is free of charge. Just don't pull a bitomat.pl and don't store the backup copy of the wallet on the same ephemeral storage as the working blockchain.

Could you give me a little bit more info on what happened?


Title: Re: How large an Amazon EC2 instance for Bitcoin?
Post by: crazy_rabbit on March 11, 2013, 07:50:19 PM
Protipp: Use a BIP32 deterministic wallet to create addresses on the fly on the server but keep the private keys offline in your wallet.

I'm building a online wallet- I would need the private keys on the server, no?


Title: Re: How large an Amazon EC2 instance for Bitcoin?
Post by: SgtSpike on March 11, 2013, 07:59:35 PM
I'm running a bitcoin service on amazon Ec2, and I need to run a full version of bitcoin- block chain included. How much disk space will I need to keep on hand to keep bitcoind from crashing? (my current problem).
The smallest has 160GB of the local storage. That is more than enough for couple of years. I/O to the local storage is free of charge. Just don't pull a bitomat.pl and don't store the backup copy of the wallet on the same ephemeral storage as the working blockchain.

Could you give me a little bit more info on what happened?
They stored the backup on the same storage as the normal.  Amazon pulled their storage for whatever reason.  They lost 25,000 BTC because they didn't have the wallet backed up to anything else, and Amazon wiped it when their storage was pulled.


Title: Re: How large an Amazon EC2 instance for Bitcoin?
Post by: 2112 on March 11, 2013, 08:12:30 PM
Could you give me a little bit more info on what happened?
Basically bitomat.pl administrator didn't read the AWS documentation. He tried to upgrade his instance which triggered erasing of the ephemeral storage and EBS volumes not marked persistent. Only his trading database files that were on a separately attached volume survived. He even purchased the Platinum-level support from Amazon, because he didn't believe that Amazon really throws away the encryption keys to the customer's data when the contract is closed. Mt.Gox acquired bitomat.pl and made customers good on BTC deposits, PLNs weren't affected. Mt.Gox still uses Polish bank relationships established by Bitomat to operate in the SEPA zone.

Recent EC2-related thread in the Newbie zone:

https://bitcointalk.org/index.php?topic=151640.0


Title: Re: How large an Amazon EC2 instance for Bitcoin?
Post by: SgtSpike on March 11, 2013, 08:19:42 PM
Protipp: Use a BIP32 deterministic wallet to create addresses on the fly on the server but keep the private keys offline in your wallet.

I'm building a online wallet- I would need the private keys on the server, no?

Building an online wallet, on Amazon EC2 is a horrible idea, this will end in disaster. Also I hope your not using TF for this would be way over his level. He won't know how to encryption the and deliver them in javascript so only the user can use the private keys. I am warning you this will end in disaster.
Why would he use TF?  TF is designing websites, not programming them.


Title: Re: How large an Amazon EC2 instance for Bitcoin?
Post by: SgtSpike on March 11, 2013, 08:29:28 PM
Protipp: Use a BIP32 deterministic wallet to create addresses on the fly on the server but keep the private keys offline in your wallet.

I'm building a online wallet- I would need the private keys on the server, no?

Building an online wallet, on Amazon EC2 is a horrible idea, this will end in disaster. Also I hope your not using TF for this would be way over his level. He won't know how to encryption the and deliver them in javascript so only the user can use the private keys. I am warning you this will end in disaster.
Why would he use TF?  TF is designing websites, not programming them.

He created his last wallet, Terracoin wallet.
Ew, remind me not to use it.


Title: Re: How large an Amazon EC2 instance for Bitcoin?
Post by: crazy_rabbit on March 11, 2013, 08:53:15 PM
I'm running a bitcoin service on amazon Ec2, and I need to run a full version of bitcoin- block chain included. How much disk space will I need to keep on hand to keep bitcoind from crashing? (my current problem).
The smallest has 160GB of the local storage. That is more than enough for couple of years. I/O to the local storage is free of charge. Just don't pull a bitomat.pl and don't store the backup copy of the wallet on the same ephemeral storage as the working blockchain.

Could you give me a little bit more info on what happened?
They stored the backup on the same storage as the normal.  Amazon pulled their storage for whatever reason.  They lost 25,000 BTC because they didn't have the wallet backed up to anything else, and Amazon wiped it when their storage was pulled.

Well that's plain silly.


Title: Re: How large an Amazon EC2 instance for Bitcoin?
Post by: crazy_rabbit on March 11, 2013, 08:54:29 PM
Protipp: Use a BIP32 deterministic wallet to create addresses on the fly on the server but keep the private keys offline in your wallet.

I'm building a online wallet- I would need the private keys on the server, no?

Building an online wallet, on Amazon EC2 is a horrible idea, this will end in disaster. Also I hope your not using TF for this would be way over his level. He won't know how to encryption the and deliver them in javascript so only the user can use the private keys. I am warning you this will end in disaster.
Why would he use TF?  TF is designing websites, not programming them.

TF thus far has proved quite adept at programming, I've been happy so far. If someone has a good reason (and proof) as to why he shouldn't be used I would take a look at it.


Title: Re: How large an Amazon EC2 instance for Bitcoin?
Post by: crazy_rabbit on March 11, 2013, 08:56:58 PM
Protipp: Use a BIP32 deterministic wallet to create addresses on the fly on the server but keep the private keys offline in your wallet.

I'm building a online wallet- I would need the private keys on the server, no?

Building an online wallet, on Amazon EC2 is a horrible idea, this will end in disaster. Also I hope your not using TF for this would be way over his level. He won't know how to encryption the and deliver them in javascript so only the user can use the private keys. I am warning you this will end in disaster.

Relax- I'm interested in hearing why something like this can't be done on EC2. And besides- for the moment it's Terracoin, so relax. :-)


Title: Re: How large an Amazon EC2 instance for Bitcoin?
Post by: crazy_rabbit on March 11, 2013, 10:28:35 PM
Protipp: Use a BIP32 deterministic wallet to create addresses on the fly on the server but keep the private keys offline in your wallet.

I'm building a online wallet- I would need the private keys on the server, no?

Building an online wallet, on Amazon EC2 is a horrible idea, this will end in disaster. Also I hope your not using TF for this would be way over his level. He won't know how to encryption the and deliver them in javascript so only the user can use the private keys. I am warning you this will end in disaster.

Relax- I'm interested in hearing why something like this can't be done on EC2. And besides- for the moment it's Terracoin, so relax. :-)

To many variables not controlled by you, if you want to do this right get a dedicated box.

But EC2 is a dedicated box- although I have to trust Amazon I suppose, and they have sophisticated levels of user permissions. Seems safer to use then a dedicated box, especially where redundancy is concerned.


Title: Re: How large an Amazon EC2 instance for Bitcoin?
Post by: crazy_rabbit on March 11, 2013, 10:29:28 PM
Protipp: Use a BIP32 deterministic wallet to create addresses on the fly on the server but keep the private keys offline in your wallet.

I'm building a online wallet- I would need the private keys on the server, no?

Building an online wallet, on Amazon EC2 is a horrible idea, this will end in disaster. Also I hope your not using TF for this would be way over his level. He won't know how to encryption the and deliver them in javascript so only the user can use the private keys. I am warning you this will end in disaster.
Why would he use TF?  TF is designing websites, not programming them.

TF thus far has proved quite adept at programming, I've been happy so far. If someone has a good reason (and proof) as to why he shouldn't be used I would take a look at it.

Go look at the money packer dude's website release the code, it was a mess. I almost threw up, he is using php to construct html LIKE SERIOUSLY, and there were some flaws in the code, which I still looking at.

You wouldn't use PHP to construct HTML? I thought that was pretty standard, unless I'm behind the times.


Title: Re: How large an Amazon EC2 instance for Bitcoin?
Post by: speeder on March 11, 2013, 10:36:01 PM
Amazon EC2 purpose is handle highly scalable services.

NOT critical services.

It was made to handle netflix, where there are no damage if the thing go down (Beside angry people).

It is not supposed to handle... bitomat.


Title: Re: How large an Amazon EC2 instance for Bitcoin?
Post by: crazy_rabbit on March 11, 2013, 10:42:44 PM
Amazon EC2 purpose is handle highly scalable services.

NOT critical services.

It was made to handle netflix, where there are no damage if the thing go down (Beside angry people).

It is not supposed to handle... bitomat.

But it seems like the same thing no? Its a server. Bitcoin needs highly scalable (look at gox!) and I think the Netflix people would think an enormous amount of damage would be done if it went down. Netflix is worth more then all of bitcoin combined.

I am sincerely interested in your opinion, but could you offer any facts to support it? Something I could read up on?


Title: Re: How large an Amazon EC2 instance for Bitcoin?
Post by: SgtSpike on March 11, 2013, 10:47:21 PM
Amazon EC2 purpose is handle highly scalable services.

NOT critical services.

It was made to handle netflix, where there are no damage if the thing go down (Beside angry people).

It is not supposed to handle... bitomat.

But it seems like the same thing no? Its a server. Bitcoin needs highly scalable (look at gox!) and I think the Netflix people would think an enormous amount of damage would be done if it went down. Netflix is worth more then all of bitcoin combined.

I am sincerely interested in your opinion, but could you offer any facts to support it? Something I could read up on?
Again, Amazon permanently lost 25k BTC because of ???.  It doesn't matter - bottom line is, data storage there should not be considered reliable or long-term.  Everything stored there should also be backed up (constantly) to another source, or the risk exists of a similar thing happening to you.  I don't think it's a bad route to go, just make sure everything is backed up, and double-check those backups quite often.


Title: Re: How large an Amazon EC2 instance for Bitcoin?
Post by: crazy_rabbit on March 11, 2013, 11:06:50 PM
Amazon EC2 purpose is handle highly scalable services.

NOT critical services.

It was made to handle netflix, where there are no damage if the thing go down (Beside angry people).

It is not supposed to handle... bitomat.

But it seems like the same thing no? Its a server. Bitcoin needs highly scalable (look at gox!) and I think the Netflix people would think an enormous amount of damage would be done if it went down. Netflix is worth more then all of bitcoin combined.

I am sincerely interested in your opinion, but could you offer any facts to support it? Something I could read up on?
Again, Amazon permanently lost 25k BTC because of ???.  It doesn't matter - bottom line is, data storage there should not be considered reliable or long-term.  Everything stored there should also be backed up (constantly) to another source, or the risk exists of a similar thing happening to you.  I don't think it's a bad route to go, just make sure everything is backed up, and double-check those backups quite often.

Always a good idea. I think what that site did, (if it didn't steal them) was just terminate their instance, which it informs you clearly wil result in them terminating your data. :-/


Title: Re: How large an Amazon EC2 instance for Bitcoin?
Post by: gweedo on March 12, 2013, 01:15:15 AM
Amazon EC2 purpose is handle highly scalable services.

NOT critical services.

It was made to handle netflix, where there are no damage if the thing go down (Beside angry people).

It is not supposed to handle... bitomat.

But it seems like the same thing no? Its a server. Bitcoin needs highly scalable (look at gox!) and I think the Netflix people would think an enormous amount of damage would be done if it went down. Netflix is worth more then all of bitcoin combined.

I am sincerely interested in your opinion, but could you offer any facts to support it? Something I could read up on?

If GOX was using EC2, they be incredibly dumb, they would have to play by amazon's rules and could lose control of the information really quickly. Lets be honest here, for the benefits of EC2 you would not get them. EC2 is great scalability horizontally.


Title: Re: How large an Amazon EC2 instance for Bitcoin?
Post by: crazy_rabbit on March 12, 2013, 07:54:30 AM
Amazon EC2 purpose is handle highly scalable services.

NOT critical services.

It was made to handle netflix, where there are no damage if the thing go down (Beside angry people).

It is not supposed to handle... bitomat.

But it seems like the same thing no? Its a server. Bitcoin needs highly scalable (look at gox!) and I think the Netflix people would think an enormous amount of damage would be done if it went down. Netflix is worth more then all of bitcoin combined.

I am sincerely interested in your opinion, but could you offer any facts to support it? Something I could read up on?

If GOX was using EC2, they be incredibly dumb, they would have to play by amazon's rules and could lose control of the information really quickly. Lets be honest here, for the benefits of EC2 you would not get them. EC2 is great scalability horizontally.

Thanks, but your still not saying why you think this. No examples? Nothing specific? Do you work for Linode?


Title: Re: How large an Amazon EC2 instance for Bitcoin?
Post by: deadweasel on June 05, 2013, 05:40:00 PM
Amazon EC2 purpose is handle highly scalable services.

NOT critical services.

It was made to handle netflix, where there are no damage if the thing go down (Beside angry people).

It is not supposed to handle... bitomat.

But it seems like the same thing no? Its a server. Bitcoin needs highly scalable (look at gox!) and I think the Netflix people would think an enormous amount of damage would be done if it went down. Netflix is worth more then all of bitcoin combined.

I am sincerely interested in your opinion, but could you offer any facts to support it? Something I could read up on?

If GOX was using EC2, they be incredibly dumb, they would have to play by amazon's rules and could lose control of the information really quickly. Lets be honest here, for the benefits of EC2 you would not get them. EC2 is great scalability horizontally.

Thanks, but your still not saying why you think this. No examples? Nothing specific? Do you work for Linode?

Go read there privacy policy.


I'm late to the party, but what if someone just wanted to run a node?   It seems pretty safe to just have bitcoind running there, w/out any btc in the wallet.  

The following will demonstrate my lack of understanding, so please criticize and inform me.



Could you run bitcoind on free EC2 instance and then add it as a node to your regular box Armory/Bitcoind as an extra node?  

Is it possible to send to this node first to get that first confirmation?  

Basically, I'd want my own node to get into a block quicker, but I don't have the understanding to know if this would help.  I doubt it would as it wouldn't speed up the other 5 confirms.....

Or I'd just like to have it to have my own copy of the blockchain to verify against, it case the damn thing forks again.


Title: Re: How large an Amazon EC2 instance for Bitcoin?
Post by: Dabs on June 06, 2013, 08:32:46 AM
I read before that Gavin used to run bitcoind on a micro Amazon instance, or the lowest priced Linode.

For my purpose, all I really want is a bitcoind that has walletnotify to get transaction information as it happens on the block chain. The wallet.dat file will be password protected by a really long password, and if that's not enough, it will not have any real private key information. Just the public keys. Care of pywallet.

FYI, pywallet can insert a bitcoin address into a wallet.dat file without the correct private key (or with the wrong one). bitcoind becomes a watching only node, that can't spend the funds it sees.

What's the cheapest service that can handle this? I just need transaction id, what addresses sent the coin, how much coins, and what block it gets confirmed in.