Optimizing bitcoind's use?

[Fking]

I have bitcoind installed on a quite powerful dedicated server.
For security reasons i've placed the front end site that uses it on separate shared hosting.
It has just some PHP scripts, mysql db and a basic site there, so I thought it would be OK BUT!

It seems that connections to bitcoind like connect checking, incoming payments checking...  take a lot of time and work slow, and with just few users using the site simultaneously we hit the "Entry processes" limit on the host (which is 20).
I'm not sure if this is common issue or it's a problem with our code?

I'm looking for ideas to solve this problem

[Fking]

any thoughts?

[markm]

Often if you buy off the rack / rackspace-ilk hosting, the number of connections the webserver is configured to allow to the mysql database does not match the number of connections the database is configured to allow.

When traffic increases, sometimes people end up only increasing one of those sides of the thing.

Entry processes sounds vaguely familiar as maybe relating to such situations.

Maybe though it might also relate to connections you try to make to another machines to talk to your coin daemon.

Like maybe your webserver, or the host running it, has some limit on outgoing connections. or the coin daemon's machine has some limit on incoming ones.

Its not really clear to me what layer is telling you there is a problem, like is it something the webserver is saying, or something mysql is saying, or something the operating system on one of the machines you are connecting between is saying.

20 connections seems a puny number to cause a problem, whatever layer is having the problem.

Oops, wait... I mean, setting your limit to 20 seems crazy low, who set it that low and why?

Aha, the shared hosting, likely! DIrt cheap hosting? Maybe even free like in my .sig ?

(Actually I think the free hosting in my .sig might not have that low a limit, I think where it starts to bite is only 5 gigs of bandwidth per month. Oodles of nice stuff, but put any eyecandy on it and you'll go over 5 gigs a month with not a lot of regular daily visitors...)

-MarkM-

[airsheep]

I've also noticed that the bitcoind JSON-RPC response times do seem a bit high, especially if running on modest hardware. But I've not investigated further.

I wonder how large scale bitcoin sites (e.g. satoshidice, mtgox) perform operations such as checking for transactions and making transactions - do they use:

• the standard bitcoind JSON-RPC interface
• a lightly patched bitcoind
• a heavily customised bitcoind
• a proprietary client that speaks the bitcoin protocol
• an application that bypasses bitcoind and pokes around in the bitcoind database directly
• a mixture of the above!

[behindtext]

I have bitcoind installed on a quite powerful dedicated server.
For security reasons i've placed the front end site that uses it on separate shared hosting.
It has just some PHP scripts, mysql db and a basic site there, so I thought it would be OK BUT!

It seems that connections to bitcoind like connect checking, incoming payments checking...  take a lot of time and work slow, and with just few users using the site simultaneously we hit the "Entry processes" limit on the host (which is 20).
I'm not sure if this is common issue or it's a problem with our code?

I'm looking for ideas to solve this problem

you should decouple your problem into smaller pieces that you can test.

make a script that runs on the same machine that is running bitcoind that performs many concurrent operations that you list (connect checking, incoming pmt check, etc). ideally this would be a shell script or something that doesn't use the php + mysql. the upshot is cut out the mysql + php stuff and test bitcoind first. if this is really slow, you may need to proxy bitcoind interactions.

after testing this, move on to the php and mysql. test php first by itself, so do not store tx in the db. this will give you info about php performance. add the mysql stuff last and you will have a pretty clear picture of what is bottlenecking.