Bitcoin Forum

Bitcoin => Development & Technical Discussion => Topic started by: pocesar on September 09, 2013, 09:03:36 PM



Title: is it possible to have datadir over network instead of disk?
Post by: pocesar on September 09, 2013, 09:03:36 PM
I'm trying to figure out a way to reuse existing blockchain database to multiple spot instances, so I don't have to download the blockchain per each spot instance everytime they go up on auto scaling on AWS. The thing is, datadir needs a disk path, how could I create a on-demand instance to keep the blockchain so each spot instance can take advantage with the lastest blockchain? would many daemons writing to the same blockchain database be a problem?

like so:

Code:
 
                                                    /--> [spot instance with daemon]
[frontend] ---> [blockchain / pool load balancer]  <-->  [spot instance with daemon]
                                                    \--> [spot instance with daemon]


Title: Re: is it possible to have datadir over network instead of disk?
Post by: justusranvier on September 09, 2013, 09:11:01 PM
In principle you should be able to use NFS to share the directory containing the blockchain.

Or you could use Bits of Proof (http://bitsofproof.com/) instead of bitcoind, which was explicitly designed for that kind of thing.


Title: Re: is it possible to have datadir over network instead of disk?
Post by: kostagr33k on September 10, 2013, 03:46:58 AM
From the research  I have done, I believe you can have one master (two for safety) that are online 100% of the time, and each secondary node "slave" off of the masters. That way, when the slave needs any information, it will query one of the masters.

If that is something that will work in your infrastructure, I would google that.


If you can't figure it out, feel free to PM me.


kosta


Title: Re: is it possible to have datadir over network instead of disk?
Post by: jdbtracker on September 12, 2013, 03:22:10 PM
Build a Hadoop cluster and distribute the hard drive over a network and connect it. It could work.


Title: Re: is it possible to have datadir over network instead of disk?
Post by: pocesar on September 13, 2013, 12:47:26 AM
the only thing that I haven't actually tried is that if the underlaying database used in bitcoind / bitcoin-qt has some sort of locking. if 2 or more daemons are connected to the network, they will (expected) receive the same info from the network, and try to work the same data to the database, right?


Title: Re: is it possible to have datadir over network instead of disk?
Post by: kostagr33k on September 13, 2013, 01:47:41 AM
Try adding this to your Slave bitcoind Node config:

Code:
rpcconnect=MasterIP 
connect = 0.0.0.0

rpcconnect Means all commands from the Slave, will "proxy" to the MAster Node

connect=0.0.0.0 Tells the Slave node not to connect to the Network and NOT download the chain.


That should let you setup Slave bitcoind Nodes that you can have throughout your network.