Hardware accelerated bitcoind?

[LiveJay]

Is it possible to use asic hardware to improve the performance of bitcoind?  Say when it's downloading the blockchain and running all those sha256 calculations?  Maybe we could use some of those usb stick miners for this.

-Jay

[1714056196]

1714056196

[Come-from-Beyond]

Is it possible to use asic hardware to improve the performance of bitcoind?  Say when it's downloading the blockchain and running all those sha256 calculations?  Maybe we could use some of those usb stick miners for this.

-Jay

To accelerate bitcoind u need to implement in hardware the signature verification algo.

[Sukrim]

Miners could quickly verify the hashes of the block chain, yes. This is not the main bottleneck however, my mobile phone can do that in seconds for the whole chain. The main work though is likely the signature checking though and there is no ASIC implementation that I know of. I am also not too sure if it would be feasible to even offload this to OpenCL (on GPUs for example) as there is a lot of data to be processed and lots of changes don't work too well with distributed algorithms (too much communication overhead creates a bottleneck there instead of at the computation end).

It might be an interesting thing to try, also for stuff like SSL or other crypto, to offload it to more specialized hardware... I am not aware of any serious initiatives in that direction though - until there is OpenSSL-OpenCL, I doubt many people will even trust something like that.

[blub]

well, the signature checking could be done on a gpu if most of the signatures are correct. then the gpu can check all signatures and only very seldom signatures have to be thrown away becaus a previous transaction was invalid.

[wtogami]

You can "hardware accelerate" Bitcoin by using a x86_64 binaries instead of 32bit.  The hashing functions and signature validation is *much* faster on 64bit.

Linux distributes 64bit and 32bit binaries.  Currently Windows and Mac builds are 32bit only and could be faster if there were 64bit builds.  The developers don't want to do additional 64bit builds because that would be more time in QA testing, so there would be a cost in paying developers to make and test it.

[RagnarDanneskjold]

You can "hardware accelerate" Bitcoin by using a x86_64 binaries instead of 32bit.  The hashing functions and signature validation is *much* faster on 64bit.

Linux distributes 64bit and 32bit binaries.  Currently Windows and Mac builds are 32bit only and could be faster if there were 64bit builds.  The developers don't want to do additional 64bit builds because that would be more time in QA testing, so there would be a cost in paying developers to make and test it.

Yes, this^^

addressing op: yes, greater computing power will process your clients activity faster, so unless you've got a few milliseconds to spare, I WOULD TAKE wtogami's advice and UPGRADE YOUR SHIT before its too late

[LiveJay]

I WOULD TAKE wtogami's advice and UPGRADE YOUR SHIT before its too late

Well, my MacBook Pro is pretty high end, but I'll look into rebuilding locally to take advantage of the 64 bit libs.  But what do you mean by "too late"?

-Jay

[RagnarDanneskjold]

I WOULD TAKE wtogami's advice and UPGRADE YOUR SHIT before its too late

Well, my MacBook Pro is pretty high end, but I'll look into rebuilding locally to take advantage of the 64 bit libs.  But what do you mean by "too late"?

-Jay

you'll find out soon enough

[wumpus]

Unless you're mining, the expensive part is ECDSA verification, not SHA256.

It is likely possible to implement it paralellized on the GPU with OpenCL, not sure though (also not how much you'd really win...), and it's not as easy as with SHA256.

According to Google at least one person tried to look into it before: http://stackoverflow.com/questions/12924059/extracting-a-part-of-openssl-into-one-single-preprocessed-source-file

It may be better to start from sipa's secp256k1 library instead of OpenSSL, as it is already more specialized.

[lophie]

http://software.intel.com/en-us/articles/intel-sha-extensions

[grue]

http://software.intel.com/en-us/articles/intel-sha-extensions

see:

Unless you're mining, the expensive part is ECDSA verification, not SHA256.

It is likely possible to implement it paralellized on the GPU with OpenCL, not sure though (also not how much you'd really win...), and it's not as easy as with SHA256.

According to Google at least one person tried to look into it before: http://stackoverflow.com/questions/12924059/extracting-a-part-of-openssl-into-one-single-preprocessed-source-file

It may be better to start from sipa's secp256k1 library instead of OpenSSL, as it is already more specialized.

[DeathAndTaxes]

As pointed out CPU is not the bottleneck.  The most cpu expensive portion of block verification is ECDSA signature verification and an average CPU today can do >15,000 verifications per second.  That likely will scale with Moore's law.   To put that into perspective the Bitcoin network right now is <0.5 tps.

Bandwidth between nodes is the largest bottleneck, after that would be IO speed, after that would be available memory, far after that is disk space and magnitudes after that would be cpu power.  Still in theory if Bitcoin every became many magnitudes larger one could design a ASIC designed to quickly verify transactions and blocks.

[lophie]

As pointed out CPU is not the bottleneck.  The most cpu expensive portion of block verification is ECDSA signature verification and an average CPU today can do >15,000 verifications per second.  That likely will scale with Moore's law.   To put that into perspective the Bitcoin network right now is <0.5 tps.

Bandwidth between nodes is the largest bottleneck, after that would be IO speed, after that would be available memory, far after that is disk space and magnitudes after that would be cpu power.  Still in theory if Bitcoin every became many magnitudes larger one could design a ASIC designed to quickly verify transactions and blocks.

Ok I see, Just a question do you see other users (including businesses) more interested in this than miners, and why?

[DeathAndTaxes]

I don't see ANYONE being interested or needing this for a very long time if ever.  As pointed out a CPU is capable of >15,000 tps (or 30,000x more tx volume that Bitcoin currently has).  Moore's law is still alive and well for the short term so a decade from now that is probably going to be more like 480,000 tps (about 50x what VISA processes).

Still maybe someday in the future it might make sense as a energy/cost saving measure but that day is likely a decade or more away (if ever).

[lophie]

But guys be fair in comparison Bitcoin will NEVER beat centralized systems for transactions. Decentralized will never beat centralized in number of transactions per second

Bitcoin wins epically in other aspects, just not this one.

[grue]

But guys be fair in comparison Bitcoin will NEVER beat centralized systems for transactions. Decentralized will never beat centralized in number of transactions per second

Bitcoin wins epically in other aspects, just not this one.

okay... but that was never the topic of this discussion 

[cunicula]

But guys be fair in comparison Bitcoin will NEVER beat centralized systems for transactions. Decentralized will never beat centralized in number of transactions per second

Bitcoin wins epically in other aspects, just not this one.

I think you could approach the efficiency of a centralized system if you used a deterministic mining process.
Then you could direct information directly at most likely possible next miner(s) rather than waiting for it to defuse through a network.
It's still decentralized as long as the "known possible next miner(s)" are constantly changing over time.

[wumpus]

As pointed out CPU is not the bottleneck.  The most cpu expensive portion of block verification is ECDSA signature verification and an average CPU today can do >15,000 verifications per second.  That likely will scale with Moore's law.   To put that into perspective the Bitcoin network right now is <0.5 tps.

Speeding up the ECDSA verification could help during the initial sync (This is one of the reasons sipa made the secp256k1 library with ECDSA operations in hand-optimized assembly).

Not that I'm expecting a GPU based implementation any time soon, but someone may be crazy enough

[Sukrim]

Yes, I also assumed that it isn't maintaining operation but bootstrapping that is the main issue for the OP.

I read somewhere about ~60 million transactions in the block chain so far - with 15k checks per second that still takes 4000 seconds or a bit more than 1 hour, even if they are beamed directly in the CPU cache with no IO latency or overhead at all. I wouldn't call that very significant (after all it is checking all transactions in Bitcoin ever) but still there might be some potential for optimization, especially seeing how much people were able to speed up computations for SHA256(SHA256()) for mining over the years and ECDSA is to my understanding not even as complex but just uses larger numbers that are not easy to handle.

secp256k1_OpenCL would be a really cool thing to have and it might be worth it to explore this direction further, even if there are no direct benefits right now.