Bitcoin Forum
December 05, 2016, 02:37:09 AM *
News: To be able to use the next phase of the beta forum software, please ensure that your email address is correct/functional.
 
   Home   Help Search Donate Login Register  
Pages: [1] 2 »  All
  Print  
Author Topic: Multiple bitcoind on one machine  (Read 5109 times)
MrJoshua
Member
**
Offline Offline

Activity: 76


View Profile
August 06, 2011, 02:04:05 AM
 #1

Is is possible to run multiple bitcoind instances on a single machine?  

I'm trying to develop some web services.  One of those services needs the ability to process unique wallet files / private keys for each user.  Currently this is impractical because the amount of time it takes to flush and relaunch bitcoind with rescan to load unique wallets.  Even though this takes time, it could be somewhat manageable if I can run a bitcoind for every active user session.  I can use multiple servers, but I'd like to be able run more then one bitcoind per server instance to make more effective use of each server's resources.

I suspect dynamic wallet switching directly in bitcoind is still a bit far off right?

Thanks,
j

The value of bitcoins is not a theory, predictions of it's failure are what is theoretical.
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1480905429
Hero Member
*
Offline Offline

Posts: 1480905429

View Profile Personal Message (Offline)

Ignore
1480905429
Reply with quote  #2

1480905429
Report to moderator
1480905429
Hero Member
*
Offline Offline

Posts: 1480905429

View Profile Personal Message (Offline)

Ignore
1480905429
Reply with quote  #2

1480905429
Report to moderator
1480905429
Hero Member
*
Offline Offline

Posts: 1480905429

View Profile Personal Message (Offline)

Ignore
1480905429
Reply with quote  #2

1480905429
Report to moderator
2112
Legendary
*
Offline Offline

Activity: 1708



View Profile
August 06, 2011, 02:19:01 AM
 #2

Is is possible to run multiple bitcoind instances on a single machine?  
Yes. Each one has to be under a different account and use "-rpcport" (documented) and "-port" (undocumented) or "-nolisten". The overhead can be stupendous though.

Please comment, critique, criticize or ridicule BIP 2112: https://bitcointalk.org/index.php?topic=54382.0
Long-term mining prognosis: https://bitcointalk.org/index.php?topic=91101.0
Furyan
Full Member
***
Offline Offline

Activity: 175



View Profile
August 06, 2011, 10:12:38 AM
 #3

Is is possible to run multiple bitcoind instances on a single machine?  
Yes. Each one has to be under a different account and use "-rpcport" (documented) and "-port" (undocumented) or "-nolisten". The overhead can be stupendous though.

Instead of "-port" you can use -connectnode and have it connect to an already-running bitcoin to do the peer communication.

And you don't have to run them on a different account, you can just use the -datadir flag to point the second instance to a different folder. It will rebuild its own block chain, or you can copy over the existing one to that data dir.  They'll have separate wallets though.

See Gavin Andresen's instructions here:

https://bitcointalk.org/index.php?topic=589.0
payb.tc
Hero Member
*****
Offline Offline

Activity: 812



View Profile
August 06, 2011, 01:07:24 PM
 #4

The overhead can be stupendous though.

You mean as in ram usage or are you talking about something else?

i was able to get 2 running successfully for a while on a test server.

I wonder what the record is for the most amount of bitcoind instances running together successfully... anyone running 3 or more?
Furyan
Full Member
***
Offline Offline

Activity: 175



View Profile
August 06, 2011, 01:26:40 PM
 #5

The overhead can be stupendous though.

You mean as in ram usage or are you talking about something else?

i was able to get 2 running successfully for a while on a test server.

I wonder what the record is for the most amount of bitcoind instances running together successfully... anyone running 3 or more?


I heard of someone running 8. I never did hear how that worked out Smiley
2112
Legendary
*
Offline Offline

Activity: 1708



View Profile
August 06, 2011, 01:59:08 PM
 #6

You mean as in ram usage or are you talking about something else?
RAM doesn't seem to be a problem. The worst are: disk access queues and disk space usage. Then there's a CPU use spikes and network traffic spikes during operation when separate daemons duplicate their work. Last but not least is the outgoing network connectivity which gets close to launching a non-distributed DoS on the servers when your multi-bitcoind machine restarts after maintenance.

Overall it isn't pretty and it would be hard to maintain and troubleshoot.

Please comment, critique, criticize or ridicule BIP 2112: https://bitcointalk.org/index.php?topic=54382.0
Long-term mining prognosis: https://bitcointalk.org/index.php?topic=91101.0
payb.tc
Hero Member
*****
Offline Offline

Activity: 812



View Profile
August 06, 2011, 02:06:05 PM
 #7

so i guess the solution is to modify bitcoind so that it can perform the functions of multiple bitcoind instances?

run several wallets at the same time, separately, all from the one binary and one set of ports.
2112
Legendary
*
Offline Offline

Activity: 1708



View Profile
August 06, 2011, 02:45:33 PM
 #8

so i guess the solution is to modify bitcoind so that it can perform the functions of multiple bitcoind instances?
run several wallets at the same time, separately, all from the one binary and one set of ports.
I think I responded to you in the PHP thread. Your only hope is that libbitcoin development group delivers something useable. The core development team is actively working against this goal. What you have here is the essence of using obfuscated C++ code for the purpose of guerrilla warfare amongst the competing software development teams. Satoshi is/was a grand-master of it.

Please comment, critique, criticize or ridicule BIP 2112: https://bitcointalk.org/index.php?topic=54382.0
Long-term mining prognosis: https://bitcointalk.org/index.php?topic=91101.0
payb.tc
Hero Member
*****
Offline Offline

Activity: 812



View Profile
August 06, 2011, 03:05:09 PM
 #9

so i guess the solution is to modify bitcoind so that it can perform the functions of multiple bitcoind instances?
run several wallets at the same time, separately, all from the one binary and one set of ports.
I think I responded to you in the PHP thread. Your only hope is that libbitcoin development group delivers something useable. The core development team is actively working against this goal. What you have here is the essence of using obfuscated C++ code for the purpose of guerrilla warfare amongst the competing software development teams. Satoshi is/was a grand-master of it.

which goal? what's wrong with one instance of bitcoind managing multiple wallets?
markm
Legendary
*
Offline Offline

Activity: 1778



View Profile WWW
August 06, 2011, 04:41:18 PM
 #10

The overhead can be stupendous though.

You mean as in ram usage or are you talking about something else?

i was able to get 2 running successfully for a while on a test server.

I wonder what the record is for the most amount of bitcoind instances running together successfully... anyone running 3 or more?


Even my quite old machines are not having any problems running multiple *coind instances all at once and all on the same user account. (bitcoind  botcoind  britcoind  cdnbitcoind  czbitcoind  devcoind  gmcbitcoind  grfbitcoind  groupcoind  multicoind  namecoind  unbitcoind ...)

-MarkM-



Browser-launched Crossfire client now online (select CrossCiv server for Galactic  Milieu)
Free website hosting with PHP, MySQL etc: http://hosting.knotwork.com/
Furyan
Full Member
***
Offline Offline

Activity: 175



View Profile
August 06, 2011, 08:46:16 PM
 #11

so i guess the solution is to modify bitcoind so that it can perform the functions of multiple bitcoind instances?
run several wallets at the same time, separately, all from the one binary and one set of ports.
I think I responded to you in the PHP thread. Your only hope is that libbitcoin development group delivers something useable. The core development team is actively working against this goal. What you have here is the essence of using obfuscated C++ code for the purpose of guerrilla warfare amongst the competing software development teams. Satoshi is/was a grand-master of it.

which goal? what's wrong with one instance of bitcoind managing multiple wallets?


I, too, am curious what motivated that comment.

I would think if the core devs were going to work against a goal, it would be mining pools.  Mining pools were NEVER what Satoshi had in mind and de-democratize the mining process.  They also concentrate computing power in attackable entities, and if a large pool goes down even for a few hours it takes a huge chunk of the available computing power with it (unless a large portion of the miners have fallback settings, but it doesn't appear to be widely used).

The GetWork function was intended for testing, but is the single method by which mining pools are even possible.  If they wanted to work against a goal - mining pools - they'd need only to disable that single function.

They haven't done so.  I tend to think the project is far beyond working against specific goals at this point and only working toward very focused ones.
2112
Legendary
*
Offline Offline

Activity: 1708



View Profile
August 06, 2011, 10:22:26 PM
 #12

which goal?
The goal of having multiple compatible implementations of the same protocol. The longer they have an effective control over the whole network in the hands of about 6 people the more time they have to monetize their investment of time into this project.

It is quite amazing to watch an open source project that had successfully prevented any competing groups from developing a compatible implementation despite the source being in the open. Subtle changes in protocol, using corner cases of exception handling to prevent running the Linux client on the majority of commercial Linux distributions, reintroducing exact bugs fixed in earlier patches, etc. The whole arsenal of tricks of hardcore purveyors of job safety.

I have to say that watching this project is a great learning experience for me.

Please comment, critique, criticize or ridicule BIP 2112: https://bitcointalk.org/index.php?topic=54382.0
Long-term mining prognosis: https://bitcointalk.org/index.php?topic=91101.0
payb.tc
Hero Member
*****
Offline Offline

Activity: 812



View Profile
August 07, 2011, 12:41:27 AM
 #13

which goal?
The goal of having multiple compatible implementations of the same protocol.

okay so you said all that because i used the phrase 'modify bitcoind'. i didn't mean fork the project, i meant it would be good if that functionality became part of the default client.
2112
Legendary
*
Offline Offline

Activity: 1708



View Profile
August 07, 2011, 01:39:30 AM
 #14

They haven't done so.  I tend to think the project is far beyond working against specific goals at this point and only working toward very focused ones.
Well, I probably cannot change your thinking. In my opinion the great part of the effective obfuscation is to appear constructive. I'll give you an example: Bitcoin protocol is a train-wreck of endianness and alignment problems. Yet while it was being developed some "core team" members were using MacOS X {Tiger,Leopard,Snow Leopard} which makes it almost trivial to locate and fix those problems.

Consider the code:
Code:
#include <stdio.h>
union {
        char b[4];
        int w;
} t = { 0x11, 0x22, 0x33, 0x44 };
int main()
{
        return printf("%08x,%i\n",t.w,(int)sizeof(void*));
}
and its compilation and invocation:
Code:
$ gcc -arch ppc -arch i386 -arch x86_64 bo.c
$ ./a.out
44332211,8
$ arch -i386 ./a.out
44332211,4
$ arch -ppc ./a.out
11223344,4
Now go search the history of this forum for discussion about the byte-order issues. Watch the mental gymnastics expended (especially by Jeff Garzik) to quash any work or discussion about this goal.

I'll bet this will be very instructive to you, even if you disagree with me overall.

Please comment, critique, criticize or ridicule BIP 2112: https://bitcointalk.org/index.php?topic=54382.0
Long-term mining prognosis: https://bitcointalk.org/index.php?topic=91101.0
error
Hero Member
*****
Offline Offline

Activity: 574



View Profile
August 07, 2011, 02:41:32 AM
 #15

They haven't done so.  I tend to think the project is far beyond working against specific goals at this point and only working toward very focused ones.
Well, I probably cannot change your thinking. In my opinion the great part of the effective obfuscation is to appear constructive. I'll give you an example: Bitcoin protocol is a train-wreck of endianness and alignment problems. Yet while it was being developed some "core team" members were using MacOS X {Tiger,Leopard,Snow Leopard} which makes it almost trivial to locate and fix those problems.

Consider the code:
Code:
#include <stdio.h>
union {
        char b[4];
        int w;
} t = { 0x11, 0x22, 0x33, 0x44 };
int main()
{
        return printf("%08x,%i\n",t.w,(int)sizeof(void*));
}
and its compilation and invocation:
Code:
$ gcc -arch ppc -arch i386 -arch x86_64 bo.c
$ ./a.out
44332211,8
$ arch -i386 ./a.out
44332211,4
$ arch -ppc ./a.out
11223344,4
Now go search the history of this forum for discussion about the byte-order issues. Watch the mental gymnastics expended (especially by Jeff Garzik) to quash any work or discussion about this goal.

I'll bet this will be very instructive to you, even if you disagree with me overall.

Completely agree in every respect. The endianness problem is Bitcoin's biggest design flaw, and IMO warrants bumping the protocol version. Of course it's going to be a lot more work to clean this up than it would have been to do it right the first time, so it's going to be very hard to find anyone who wants to do it.

15UFyv6kfWgq83Pp3yhXPr8rknv9m6581W
Steve
Hero Member
*****
Offline Offline

Activity: 868



View Profile WWW
August 07, 2011, 12:51:11 PM
 #16

The overhead can be stupendous though.

You mean as in ram usage or are you talking about something else?

i was able to get 2 running successfully for a while on a test server.

I wonder what the record is for the most amount of bitcoind instances running together successfully... anyone running 3 or more?


I've run as many as 6 on a machine...two that are on a private local testnet...one that is on a prive testnet between several machines, one on the public test net and another couple on the real net.  Works fine, albeit the machine can get a little sluggish.  I do typically run 2 to 4 (2 on a private local testnet, one no a shared private testnet, and one on the real net).

(gasteve on IRC) Does your website accept cash? https://bitpay.com
doldgigger
Full Member
***
Offline Offline

Activity: 170


View Profile
August 07, 2011, 07:20:08 PM
 #17

I would think if the core devs were going to work against a goal, it would be mining pools.  Mining pools were NEVER what Satoshi had in mind and de-democratize the mining process.  They also concentrate computing power in attackable entities, and if a large pool goes down even for a few hours it takes a huge chunk of the available computing power with it (unless a large portion of the miners have fallback settings, but it doesn't appear to be widely used).
This is not the responsibility of the core devs, it is the responsibility of every bitcoin user. However, they are probably a natural effect of bitcoin's evolution - in other areas of P2P computing, you will also notice that evolution towards efficiency often leads to some kind of hub nodes. For bitcoin, I could imagine even more areas beyond mining where this might happen in the future.

But mining pools are not necessarily dangerous if we regulate them well. I have already started discussing about revitalizing the transaction fee market in presence of mining pools some time ago (see http://bitcointalk.org/index.php?topic=28309.0). I'd be glad to discuss further possibilities.

19orEcoqXQ5bzKbzbAnbQrCkQC5ahSh4P9
Feel free to PM me for consulting and development services.
c_k
Donator
Sr. Member
*
Offline Offline

Activity: 242



View Profile
August 08, 2011, 10:02:39 AM
 #18

OP: what is wrong with the existing functionality of Accounts in wallets?

https://en.bitcoin.it/wiki/Accounts_explained

payb.tc
Hero Member
*****
Offline Offline

Activity: 812



View Profile
August 08, 2011, 10:15:30 AM
 #19

OP: what is wrong with the existing functionality of Accounts in wallets?

https://en.bitcoin.it/wiki/Accounts_explained

how about a lack of sub-accounts and sub-sub-accounts?

just one reason of many, i'd say.
c_k
Donator
Sr. Member
*
Offline Offline

Activity: 242



View Profile
August 08, 2011, 10:34:20 AM
 #20

just one reason of many, i'd say.

Do share the other reasons as they come to you.

I don't see how what you've said so far is a problem.

Don't take me the wrong way, I am genuinely interested in where the existing functionality fails to suffice Smiley

Pages: [1] 2 »  All
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!