PetaFLOPS and how it relates to Bitcoin

[kjj]

Comparisons between integer and floating point operations are meaningless.  Not just inaccurate, or merely approximate, but not even in the same universe.

Just the Xeon CPUs in that Dutch thing are capable of finding a block every 30 hours or so, or about 40 BTC per day.  Presumably, that is more than 12 euros.

[cjp]

1) 58.2 TFlop is pathetically weak for a super computer.  A 5970 graphics card has about 4.6 TFLOPs.  So it is roughly equal to the floating point math calculations of ~12 5970 GPUs (4x that if it is double precision).

3) The largest super computers in the world is 11,280 TFLOPs.  http://i.top500.org/system/177232

That seems to be an explanation, although it seems weird that you can already create a more powerful (in terms of flops) machine with a number of GPUs that is affordable for a hobbyist. Maybe raw processing speed is not the most expensive factor in supercomputers anymore? That weather forecast thing also has a huge amount of RAM, and probably some high-speed inter-computer connections as well.

Comparisons between integer and floating point operations are meaningless.  Not just inaccurate, or merely approximate, but not even in the same universe.

I disagree with that. You can always emulate floating point operations on top of integer operations, or integer operations on top of floating point operations. This places an upper and lower bound on the speed ratio between floating point and integer operations, which is basically independent of the hardware you are using. I don't really know how big the gap is between the upper and lower bound, but at least I can say from experience that emulated floating point can be relatively fast(*).

Just the Xeon CPUs in that Dutch thing are capable of finding a block every 30 hours or so, or about 40 BTC per day.  Presumably, that is more than 12 euros.

I used an exchange rate of about 4EUR/BTC, so your value would correspond to about 160 EUR/day.

If you are correct, then how is that possible? Wouldn't they be able to have more FLOPs than they report even by doing emulated floating points on the Xeons?

Or does bitcoinwatch.com use an unrealistic estimation of the FLOPS / hashes ratio? I see in an earlier post that they use 1 INTOP = 2 FLOP. So it assumes integer is actually slower than floating point? But even if it is 4 INTOP = 1 FLOP for floating point emulation, you'd still only lose a factor 8, which is less than needed to explain the ratio between 12 EUR/day and 160 EUR/day.

(*) I did some speed measurements on an ARM 7 without floating point unit and a Pentium 3 or 4 with floating point unit. On the ARM, emulated floating point operations (addition and multiplication) were actually faster w.r.t. integer operations than the hardware-accelerated floating point w.r.t. integer on the Pentium (TfloatARM / TintARM < TfloatIntel / TintIntel). I did not use things like SSE on the Pentium.

[DeathAndTaxes]

That seems to be an explanation, although it seems weird that you can already create a more powerful (in terms of flops) machine with a number of GPUs that is affordable for a hobbyist. Maybe raw processing speed is not the most expensive factor in supercomputers anymore? That weather forecast thing also has a huge amount of RAM, and probably some high-speed inter-computer connections as well.

That is it.  A hobbyist GPU "Supercomputer" made up of mining rigs would only be useful for problems which have no inter node depndencies, little bandwidth, and no storage requirements.  Essentially completely useless for anything other than things like Bitcoin hashing or password cracking.

Most problems in the world are complex and weather modeling isn't hashing an incrementing nonce value each of the nodes is doing work which depends on other nodes.   A mining rig would simply fail at that.  Still it is strange they didn't use GPU in the super computer to boost the more computationally intensive portions.

I used an exchange rate of about 4EUR/BTC, so your value would correspond to about 160 EUR/day.

If you are correct, then how is that possible? Wouldn't they be able to have more FLOPs than they report even by doing emulated floating points on the Xeons?

Or does bitcoinwatch.com use an unrealistic estimation of the FLOPS / hashes ratio? I see in an earlier post that they use 1 INTOP = 2 FLOP. So it assumes integer is actually slower than floating point? But even if it is 4 INTOP = 1 FLOP for floating point emulation, you'd still only lose a factor 8, which is less than needed to explain the ratio between 12 EUR/day and 160 EUR/day.

(*) I did some speed measurements on an ARM 7 without floating point unit and a Pentium 3 or 4 with floating point unit. On the ARM, emulated floating point operations (addition and multiplication) were actually faster w.r.t. integer operations than the hardware-accelerated floating point w.r.t. integer on the Pentium (TfloatARM / TintARM < TfloatIntel / TintIntel). I did not use things like SSE on the Pentium.

There is no Universal ratio.  The ratio they used is relevant only for AMD 5000 series GPUs.  On any other hardware it is completely off and not by 10%, or 50% but 1000% or 20,000%.  It is really ony a guestimate and shouldn't be taken as a serious value.  Nobody rates supercomputers in intops.  If they did one could simply post the intops of the network and be done.

The * which should be next to the Petaflop numbers should read something like:
"(if the entire network consisted of x000 AMD HD 5870 GPUs.  Single precision only, double precision would be 1/4th stated speed).

[maaku]

Comparisons between integer and floating point operations are meaningless.  Not just inaccurate, or merely approximate, but not even in the same universe.

I disagree with that. You can always emulate floating point operations on top of integer operations, or integer operations on top of floating point operations. This places an upper and lower bound on the speed ratio between floating point and integer operations, which is basically independent of the hardware you are using. I don't really know how big the gap is between the upper and lower bound, but at least I can say from experience that emulated floating point can be relatively fast(*).

That's not what's at issue. FLOPS is not, in fact, FLoating Point Operations Per Second but the numeric result of running a very specific, standardized benchmark from the LINPACK codes solving a large, dense system of linear equations. It is a benchmark that is meaningful for most scientific and technical disciplines, but says *nothing* about the ability to crank out SHA-256 hashes.

LINPACK, which is what measures FLOPS, is a test of general-purpose scientific/technical capability. Bitcoin is highly, HIGHLY specialized. Using a measurement of one to derive the other could be off by orders of magnitude, not even in the same ballpark. It's a totally meaningless, apples-to-oranges comparison.

[mrb]

That's not what's at issue. FLOPS is not, in fact, FLoating Point Operations Per Second but the numeric result of running a very specific, standardized benchmark from the LINPACK codes solving a large, dense system of linear equations. It is a benchmark that is meaningful for most scientific and technical disciplines, but says *nothing* about the ability to crank out SHA-256 hashes.

FLOPS is floating point operations per second. LINPACK FLOPS are just meaningful in the context of the TOP500 supercomputer rank because this organization decided to use this benchmark to establish the rank. A supercomputer might achieve "x" FLOPS on linear equations with LINPACK, "y" FLOPS when doing protein folding, and "z" FLOPS when doing some other work. The "x" FLOPS value of the LINPACK benchmark is no more significant than "y" or "z".

All are a fraction of "t", the theoretical peak FLOPS of the hardware (which can often be reached within 1-2% with a useless loop of multiply-add instructions). This theoretical peak can be used to predict the performance of SHA256-based Bitcoin mining because it scales linearly with the peak theoretical integer performance of a chip, which is itself directly related to its peak theoretical FLOPS performance by a fixed ratio. For example, there is exactly a 1:4 ratio between the number of integer and double precision floating point instructions that an HD 69xx series GPU can execute.

[kjj]

Why in the world are so many people so insistent on this meaningless comparison?  Floating point and integer operations aren't even done on the same hardware, which is easy to overlook since both units have been routinely packaged inside the same die for the last couple of decades.

[DanielleEber]

Still it is strange they didn't use GPU in the super computer to boost the more computationally intensive portions.

The top two current supercomputers do, in fact, use APU accelerators: http://www.top500.org/blog/lists/2013/06/press-release/

The Tianhe-2 (~34 PetaFlops) uses 48,000 Intel Xeon Phi co-processors, with 63 cores each.  The Phi is designed similar to a graphics card, but instead of the cores being optimized for graphics pipeline, they are optimized for general purpose math.  Titan (~17.5 Petaflops) uses 256K x Nvidia K20x cards, which uses the same GK-110 graphics chip as their top end consumer card.  The only real difference is the scientific card uses error correcting memory, and the consumer card doesn't, because dropping a few bits here and there in rendering a video game doesn't matter, but in doing scientific problems it does.

[oatmo]

Still it is strange they didn't use GPU in the super computer to boost the more computationally intensive portions.

The top two current supercomputers do, in fact, use APU accelerators: http://www.top500.org/blog/lists/2013/06/press-release/

The Tianhe-2 (~34 PetaFlops) uses 48,000 Intel Xeon Phi co-processors, with 63 cores each.  The Phi is designed similar to a graphics card, but instead of the cores being optimized for graphics pipeline, they are optimized for general purpose math.  Titan (~17.5 Petaflops) uses 256K x Nvidia K20x cards, which uses the same GK-110 graphics chip as their top end consumer card.  The only real difference is the scientific card uses error correcting memory, and the consumer card doesn't, because dropping a few bits here and there in rendering a video game doesn't matter, but in doing scientific problems it does.

Titan has 22,000 (not 256K) GK-110s. At best those would produce about 500MH/s each. Therefore Titan could generate 11TH/s if they chose to use it for that. I think they are too busy simulating nuclear bomb blasts to worry about Bitcoin. If they are really worried, it would be cheaper for them to do something else to mess with it...