newguy05 (OP)
|
|
January 19, 2014, 11:22:35 PM |
|
Hi guys, total newbie to amazon's AWS EC2 instances, but am very familiar with linux. Have some questions if anyone can help, thanks!
I have setup a micro EC2 instance using amazon's free trial offer (installed amazon's ami linux), downloaded and have bitcoind running on the server. I can see in the log it's syncing up. Here are my questions:
1) From my understanding, EC2 instance is not perm, as soon as it shuts down everything is wiped. Which means as soon as it shuts down all the data from bitcoind is wiped and next time i have to resync from block 1. I must not be using the right tool (EC2), can someone point me in the right direction on what exactly i should be using to fix this? I just want a regular/vps linux server that doesn't wipe the data every time it goes down.
2) Bitcoind is doing massive amount of sync on the network, i can see the network spike up in amazon's monitoring tool, is there an easy way to tell how much in/out network data i have used vs the threshold before i get charged. I clicked on all their links/monitoring but can't figure out where they keep this info. I plan to leave this running in the background but don't want to get hit with a surprise bill a week later.
Thanks!
|
|
|
|
cr1776
Legendary
Offline
Activity: 4214
Merit: 1313
|
|
January 19, 2014, 11:52:06 PM |
|
You may want to look at the ec2 ebs which is persistent. I use it for two of my servers (not bitcoin) an for 6 years now have been happy with its reliability.
As far as usage goes, I am not sure off the top of my head it has been so long.
|
|
|
|
newguy05 (OP)
|
|
January 20, 2014, 04:20:29 AM |
|
You may want to look at the ec2 ebs which is persistent. I use it for two of my servers (not bitcoin) an for 6 years now have been happy with its reliability.
As far as usage goes, I am not sure off the top of my head it has been so long.
thanks! looks like by default it is already running on EBS, it auto mounted 8 GB of space from EBS when the EC2 instance was created. Now just waiting for the block chain to catch up. It is taking forever and already used up 3GB of space. I wonder if this will be the major flaw of bitcoins, say 20 years from now, how the heck on earth will any new client able to catch up, it will take weeks and tons of space.
|
|
|
|
kostagr33k
|
|
January 20, 2014, 05:23:02 AM |
|
I actually did this last week. It took days, and I kept blowing away my bitcoind while tweaking things.
I eventually gave in and downloaded the torrent for the recent blocks, but even that took a long time to import, as Disk IO on the micro instances is very limited as well as CPU instances ... Note that the Micro instance is not guaranteed performance as it is their "free" tier"
I eventually went to a m1.large for the two day period to sync all blocks, and now seems to be working fine on the Micro instance once it caught up and I reverted.
kosta
|
|
|
|
cr1776
Legendary
Offline
Activity: 4214
Merit: 1313
|
|
January 20, 2014, 08:53:49 AM |
|
You'll want your own EBS to store the blockchain and it will be persistent. Amazon has lots of information detailing it.
|
|
|
|
thenoblebot
|
|
January 20, 2014, 11:56:09 AM |
|
Whoa .. please make sure you do not exceed the "free tier" limit for bandwidth. Cause then you are charged. Apparently its not all that "free". Check the limits or you might be surprised later.
|
|
|
|
newguy05 (OP)
|
|
January 20, 2014, 05:04:39 PM |
|
I actually did this last week. It took days, and I kept blowing away my bitcoind while tweaking things.
I eventually gave in and downloaded the torrent for the recent blocks, but even that took a long time to import, as Disk IO on the micro instances is very limited as well as CPU instances ... Note that the Micro instance is not guaranteed performance as it is their "free" tier"
I eventually went to a m1.large for the two day period to sync all blocks, and now seems to be working fine on the Micro instance once it caught up and I reverted.
kosta
Thanks guys, Kosta do you know the approx disk space requirement after bitcoind is synced up? I get 30 gb ebs for free, so unmounted the default 8gb and created a new 21 gb mount to use for bitcoind, hope that is enough space. This to me is one of the biggest shortfalls of bitcoin as time goes on, imagine 30 years from now and when bitcoin actually becomes popular, you almost need to buy hardware with all the blocks preloaded otherwise no retail users can ever catch up.
|
|
|
|
newguy05 (OP)
|
|
January 20, 2014, 06:24:22 PM |
|
ah crap i am already getting billed by amazon for I/O charges and i am only at the 9M difficulty level, this may not be such a bright idea after all, probably just going to stick with regular dedicated vps for 20 bucks a month
EBS $0.00 for the first 2 million I/O requests under monthly free tier 2,000,000 IOs(used) $0.00(charge) $0.10 per 1 million I/O requests 2,393,049 IOs(used) $0.24(charge) Total: $0.24
|
|
|
|
thenoblebot
|
|
January 22, 2014, 10:35:51 AM |
|
ah crap i am already getting billed by amazon for I/O charges and i am only at the 9M difficulty level, this may not be such a bright idea after all, probably just going to stick with regular dedicated vps for 20 bucks a month
EBS $0.00 for the first 2 million I/O requests under monthly free tier 2,000,000 IOs(used) $0.00(charge) $0.10 per 1 million I/O requests 2,393,049 IOs(used) $0.24(charge) Total: $0.24
told ya not everything is "free" in the "Free tier".
|
|
|
|
e4xit
|
|
January 22, 2014, 04:22:02 PM |
|
--snip-- This to me is one of the biggest shortfalls of bitcoin as time goes on, imagine 30 years from now and when bitcoin actually becomes popular, you almost need to buy hardware with all the blocks preloaded otherwise no retail users can ever catch up. --snip--
Really? Do you not think connection speeds will increase? That seems to me like it might be some serious short-sightedness on you part if I'm honest... Luckily the architect did not envision that **every** user requires a full copy of the blockchain, only enough to keep it decentralised. Prefessionals, hobbyists and those with the spare resources may keep full purchase ledgers, while "average Joe's" will use light/SPV clients. /topic derailment. Also, nice work hosting a bitcoind instance on EC2!
|
Not your keys, not your coins. CoinJoin, always.
|
|
|
|
newguy05 (OP)
|
|
January 23, 2014, 04:41:19 PM |
|
interesting i will take a look, yeah i looked at google app engine first but it's not a true vps like aws and has limitations in place that prevent bitcoind to run easily without some major tinkering. In the end i think a regular vps is still my best(economical) option to keep bicoind running 24/7. Something like linode that cost $20 a month and not have to worry about i/o limits etc...
|
|
|
|
thenoblebot
|
|
January 25, 2014, 08:01:39 AM |
|
interesting i will take a look, yeah i looked at google app engine first but it's not a true vps like aws and has limitations in place that prevent bitcoind to run easily without some major tinkering. In the end i think a regular vps is still my best(economical) option to keep bicoind running 24/7. Something like linode that cost $20 a month and not have to worry about i/o limits etc... Which vps do you use ? cause other server options like digital ocean, rackspace, etc have bandwidth limitations.
|
|
|
|
kostagr33k
|
|
January 25, 2014, 06:50:58 PM |
|
Sorry newguy05. Its about 16gb for the blocks folder alone: root@i .bitcoin]# df -h Filesystem Size Used Avail Use% Mounted on /dev/xvde1 30G 19G 11G 65% / none 296M 0 296M 0% /dev/shm [root@i .bitcoin]# du -chs 17G . 17G total Kosta I actually did this last week. It took days, and I kept blowing away my bitcoind while tweaking things.
I eventually gave in and downloaded the torrent for the recent blocks, but even that took a long time to import, as Disk IO on the micro instances is very limited as well as CPU instances ... Note that the Micro instance is not guaranteed performance as it is their "free" tier"
I eventually went to a m1.large for the two day period to sync all blocks, and now seems to be working fine on the Micro instance once it caught up and I reverted.
kosta
Thanks guys, Kosta do you know the approx disk space requirement after bitcoind is synced up? I get 30 gb ebs for free, so unmounted the default 8gb and created a new 21 gb mount to use for bitcoind, hope that is enough space. This to me is one of the biggest shortfalls of bitcoin as time goes on, imagine 30 years from now and when bitcoin actually becomes popular, you almost need to buy hardware with all the blocks preloaded otherwise no retail users can ever catch up.
|
|
|
|
coinrevo
Member
Offline
Activity: 70
Merit: 10
|
|
January 25, 2014, 07:12:23 PM |
|
at digitalocean you can run a node for 5$/month (with added swap space). probably cheaper.
|
|
|
|
thenoblebot
|
|
January 25, 2014, 08:31:05 PM |
|
at digitalocean you can run a node for 5$/month (with added swap space). probably cheaper.
would that be enough for a fully functional node with all the normal traffic of a website ? I believe they offer 1 TB of traffic with $5/month package.
|
|
|
|
lauren2014
Newbie
Offline
Activity: 39
Merit: 0
|
|
January 26, 2014, 12:40:45 AM |
|
at digitalocean you can run a node for 5$/month (with added swap space). probably cheaper.
i agree, aws is expensive sometimes, and not that much more reliable. ive had many websites down on aws
|
|
|
|
|
Zeal0t
Member
Offline
Activity: 78
Merit: 10
To the moon?
|
|
January 26, 2014, 02:35:16 AM |
|
I had no idea Amazon has cloud computing services, wow. Thanks for the info!
|
I vow never to use this space for sleazy referrals, gambling, spam, or to beg for handouts.
|
|
|
U1TRA_L0RD
Full Member
Offline
Activity: 126
Merit: 100
CAUTION: Angry Man with Attitude.
|
|
January 26, 2014, 10:48:47 AM |
|
I had no idea Amazon has cloud computing services, wow. Thanks for the info!
It sucks... haha, Even google has cloud services which are way better if you ask me.
|
|
|
|
|