PetaFLOPS and how it relates to Bitcoin

[johnj]

I think this issue has come up before, but I'm not very technical and could never discern a definitive answer:

Super computers which are measured in FLOPs, how much of a threat are they to Bitcoin?  I've read that hashing in BTC involves 0 flops, but then I read something about integer processing is easier than FLOPs. Hah as you can tell... I'm not very technical.

However I would like to start a discussion on super-computers and how they relate to bitcoin, and hopefully compile the data into a laymans Q&A guide.

[1714063661]

1714063661

[1714063661]

1714063661

[1714063661]

1714063661

[johnj]

Okay, so I found this,

This is exactly how bitcoinwatch estimates FLOPS.  I am not saying it is an acurate way to do the estimate, all I am saying is this is how they do it for what it is worth and for comparison to your other methods.  Many here among us question the valididty of this estimation method.

The page simply uses the following assumptions/estimates:

    1 INTOP = 2 FLOP
    1 hash = 6.35K INTOP
    1 hash = 12.7K FLOP

So the hashrate in TeraFLOP/s is simply 12.7 times the hashrate in Gigahashes/s.

As an example:  11,558.55 Gigahashs/s * 12.7 TeraFLOP/Gigahash = 146,794 TeraFLOP/s = 146 PetaFLOP/s

Is this accurate?  What 'disclaimers' would need to be in front of this if used as a reference?

[DeathAndTaxes]

Okay, so I found this,

This is exactly how bitcoinwatch estimates FLOPS.  I am not saying it is an acurate way to do the estimate, all I am saying is this is how they do it for what it is worth and for comparison to your other methods.  Many here among us question the valididty of this estimation method.

The page simply uses the following assumptions/estimates:

    1 INTOP = 2 FLOP
    1 hash = 6.35K INTOP
    1 hash = 12.7K FLOP

So the hashrate in TeraFLOP/s is simply 12.7 times the hashrate in Gigahashes/s.

As an example:  11,558.55 Gigahashs/s * 12.7 TeraFLOP/Gigahash = 146,794 TeraFLOP/s = 146 PetaFLOP/s

Is this accurate?  What 'disclaimers' would need to be in front of this if used as a reference?

That is a good rule of thumb but there is no exact metric like 12 inches = 1 foot. The exact ratio between integer performance and floating point performance depends on the chip architecture, other hardware factor, and operating system.

[johnj]

Okay, so I found this,

This is exactly how bitcoinwatch estimates FLOPS.  I am not saying it is an acurate way to do the estimate, all I am saying is this is how they do it for what it is worth and for comparison to your other methods.  Many here among us question the valididty of this estimation method.

The page simply uses the following assumptions/estimates:

    1 INTOP = 2 FLOP
    1 hash = 6.35K INTOP
    1 hash = 12.7K FLOP

So the hashrate in TeraFLOP/s is simply 12.7 times the hashrate in Gigahashes/s.

As an example:  11,558.55 Gigahashs/s * 12.7 TeraFLOP/Gigahash = 146,794 TeraFLOP/s = 146 PetaFLOP/s

Is this accurate?  What 'disclaimers' would need to be in front of this if used as a reference?

That is a good rule of thumb but there is no exact metric like 12 inches = 1 foot. The exact ratio between integer performance and floating point performance depends on the chip architecture, other hardware factor, and operating system.

Okay, so this would follow Moores law, roughly?

Edit: Again I understand that it's hard to pin down, but if it depends on hadware, and hardware follows Moores law, then if I were explaining this to someone afraid that a super-computer can just overtake bitcoin, referencing moores law for during an on-the-fly conversation shouldn't detract from the point...

Again, thanks for the clarification.

[DeathAndTaxes]

Probably faster than Moore's law.  Moore's Law indicates that Petaflops per $ would grow roughly 40% per year.

For Bitcoin the network hashing power has grown faster than Moore's law.  It is a function of the computing power of one chip * number of chips.   The number of chips mining is directly related to the value of the currency which is related to adoption.  Thus as long as the Bitcoin economy continues to grow it would be more like: 

(BitcoinGrowthRate)(Moore's Law Annualized Rate) = (Total Annual Hashing Power Growth Rate)

For things like Supercomputers they also tend to grow faster than Moores law because the average number of nodes per SuperComputer is also growing, which creates a compounding effect like Bitcoin.  By spending more you can effectively exceed Moore's law.

[johnj]

Probably faster than Moore's law.  Moore's Law indicates that Petaflops per $ would grow roughly 40% per year.

For Bitcoin the network hashing power has grown faster than Moore's law.  It is a function of the computing power of one chip * number of chips.   The number of chips mining is directly related to the value of the currency which is related to adoption.  Thus as long as the Bitcoin economy continues to grow it would be more like: 

(BitcoinGrowthRate)(Moore's Law Annualized Rate) = (Total Annual Hashing Power Growth Rate)

For things like Supercomputers they also tend to grow faster than Moores law because the average number of nodes per SuperComputer is also growing, which creates a compounding effect like Bitcoin.  By spending more you can effectively exceed Moore's law.

Okay, gotcha.

So it may go something like

Q:  But that 8 petaflop supercomputer in Japan can just over take Bitcoin?
A:  No, because current BTC network is far far greater than 8 petaflops. <Insert math here>
Q:  But what about in the future when when supercomputers can get bigger?
A:  So can Bitcoin

[DeathAndTaxes]

Exactly.  I don't think people realize exactly how powerful the Bitcoin network is.  As long as the underlying "Bitcoin economy" remains healthy then there is little risk of an outside 51% attack. 

The largest risk would be from either a malicous pool operator OR someone taking control of the pool.   Take deepbit for example.  Even if it is <51% it is still a risk.  Why?  An attacker could use Deepbit PLUS some extra hashing power to push them over the 51% mark.

But wait Tycho is a honest guy.  He may be but the cornerstone of Bitcoin is no trusted 3rd party.  If you need to trust Tycho then it isn't a trust free network.  Worse while Tycho may be a "nice guy" he may also have a family.  If Bitcoin ever reached VISA level transaction volume does anyone think it is impossible to imagine a scenario where some organized crime takes a family member to "encourage" Tycho to perform a 51% attack or "Finney attack".

The largest risk from Bitcoin comes internally.  Hopefully with time technology like p2pool will make that risk obsolete.

[Gabi]

Time to explain some things

Supercomputers are rated in PFLOPS but that is just their computing speed. Now, Bitcoin network is composed by GPU. They have a VERY HIGH FLOPS (and integer and what else) speed but that apply ONLY for very parallelizable things that can use ALL the compute shaders of a GPU

Let's consider for example a scientific computing, one that require a lot of memory and big cache and that cannot be parallelized. On a GPU it would run at snail speed, being able to use only a handful of that thousands of compute shaders a GPU have. And it would be slowed even more by memory (and cache)problem. (that's why Litecoin sucks on GPU, because it require a bigger cache)

So supercomputers not only have PetaFLOPS but they also have processors with enough cache and memory to allow them to process scientific things at full speed.


That's why a cpu "sucks" on bitcoin when compared to a gpu... the cpu do not sucks, it's only that bitcoin can fully exploit all the compute shaders of a GPU.



So, for example now bitcoin have 106PetaFLOPS of power (at least, so say bitcoin watch).

If you put enough supercomputers together and have 106 petaflops or more, can you do a 51% attack? Yes.
Is it a good idea to use supercomputers for such attack? NO! It's an epic waste of hardware and energy! Why? Because you can simply buy GPUs, spend like 1% or less and have the same computing power. Supercomputers are for things that have different requirements, that's why they have so much ram, memory and processors.


Can someone do a 51% attack without spending too much? Damn, YES! You only need like 1 million of $ or so to buy enough GPU to have 106PetaFLOPS.

[johnj]

Can someone do a 51% attack without spending too much? Damn, YES! You only need like 1 million of $ or so to buy enough GPU to have 106PetaFLOPS.

From my brief stint with the calculator, if the average GPU does ~200mh/s (midgrade ATI), and if there is ~7th/s on the network, that's a total of 35000 GPU's.  At ~$200 per GPU, that's $7,000,000 in GPU's alone.  Lets say for funsies, double that for the extra hardware you'd need.  Then there is housing, cooling, electricity, etc.... so (very roughly) lets say ~$18m total to gain a 51% foothold.

Have I messed up my 0's somewhere along the way?

[Gabi]

Oh well ok, 7 millions. It's still less than 10.

It's almost nothing for any rich guy or a government...

[DeathAndTaxes]

Nope your right.

Also at say 1.5 MH/W it would require 7.3MW electrical connection to power that hashing farm and roughly 35% more to cool it so ~10MegaWatts.  At $0.10 per kWh that would be ~$24,000 per day in electrical costs.   If racked up into 4 high shelve with 6 GPU per motherboard and 2 feet of space between shelves it would still take roughly 8,000 sq ft of warehouse space.  

So yeah it can be done but the guestimate of $1M isn't even close.

[johnj]

Nope your right.

Also at say 1.5 MH/W it would require 7.3MW electrical connection to power that hashing farm and roughly 35% more to cool it so ~10MegaWatts.  At $0.10 per kWh that would be ~$24,000 per day in electrical costs.   If racked up into 4 high shelve with 6 GPU per motherboard and 2 feet of space between shelves it would still take roughly 8,000 sq ft of warehouse space.  

So yeah it can be done but the guestimate of $1M isn't even close.

Okay, just a few more details and I think I'll have enough to compile all of this:

Any idea on how much the labor would cost to organize something like that?

Also, what how much time does it take to start reorganizing the blockchain before you put it out in the wild? (I don't even know if I'm using the right words there, sorry)

Lastly, I know the 'usual' confirmation window is 6, is 120 'absolutely' confirmed? IE no risk of 51%?  Or does it depend on the depth of the reorganized blockchain?

[Gabi]

It's winter, you want cold? Open a window. Also if we speak about rich people i expect they already have enough space for the hardware.
Ok, 1 million is not enough but 10 are.

[DeathAndTaxes]

Nope your right.

Also at say 1.5 MH/W it would require 7.3MW electrical connection to power that hashing farm and roughly 35% more to cool it so ~10MegaWatts.  At $0.10 per kWh that would be ~$24,000 per day in electrical costs.   If racked up into 4 high shelve with 6 GPU per motherboard and 2 feet of space between shelves it would still take roughly 8,000 sq ft of warehouse space.  

So yeah it can be done but the guestimate of $1M isn't even close.

Okay, just a few more details and I think I'll have enough to compile all of this:

Any idea on how much the labor would cost to organize something like that?

Those are harder to guestimate but here is a stab.

Labor wouldn't be too much.  Likely you would have a core team of say dozen linux admins who would manage the farm and write a custom distro which links to a mangement server.  For the physical work you are talking about 6000 computers which need to be assembled, booted, tested.  Even at an hour a piece thats only 6000 man hours.  50 people could likely do that in a month including all other ancillary work like creating raking and power distributions.  Maybe $200K in labor for a month.

Finding a building that can handle 7.3MW of heat and 10 MW of electrical load is a little more tough.  Worst case scenario your looking a custom AC install and something that can handle 7.3MW of heat isn't going to be cheap.  Easily in the million plus range.  Most mains even in light industrial/warehouse buildins only handle 300A @ 208V three phase which is ~180 kW so 10 MW is serious power.  No idea how hard it would be to fine and get that kind of setup installed and how many places can even hanlde that kind of localized load on the local distribution grid.

Of course you likely could break it up into say 10 teams/buildings of 4000 GPU each.

Also, what how much time does it take to start reorganizing the blockchain before you put it out in the wild? (I don't even know if I'm using the right words there, sorry)

There is no cost effective way to work backwards unless you have significantly more than 51% hashing power so you can "catch up".  Assumming the attacker starts at the current block they pretty much are guaranteed to be the longest chain after a dozen blocks or so they attack itself would happen very quickly.  The attacker would keep their bad chain private until it was sufficiently longer than the good chain to ensure the good chain can never catch up.  Then they would release it.  The warning sign would be a split blockchain and the split portion growing very rapidly.  However at that point it is too late.

Lastly, I know the 'usual' confirmation window is 6, is 120 'absolutely' confirmed? IE no risk of 51%?  Or does it depend on the depth of the reorganized blockchain?

No even 120 blocks isn't guaranteed confirmed.  Technically right now a 51% attack could have started >120 blocks ago and they have a longer "bad chain" they are keeping private.  If they released it right now it would quickly replicate through the network and replace all blocks going back to the start of the split.  The only thing that is guaranteed is a checkpoint.  Hardcoding a block # and hash into the client. the client will reject any blocks prior to that which don't match the hardcoded hash.  As a result any transaction prior to the checkpoint can't be replaced. This is possible in Bitcoin network but I don't know if it has ever been done.

[DeathAndTaxes]

It's winter, you want cold? Open a window. Also if we speak about rich people i expect they already have enough space for the hardware.
Ok, 1 million is not enough but 10 are.

Um I don't think you understand how much heat 7.3MW is.  It is enough to melt steel.  Natural convection would be insufficient to transfer that heat "out the window".  The cards near the center of the "farm" would burnup.  There is a good chance the entire building would catch fire.  There is a reason why datacenters are airconditioned year round even in say Finland and they don't even pull 1 MW.

I would say maybe $20M would be enough when you consider hardware, building space, cooling, labor, power distribution, etc.  Granted it can be done but it is hardly "cheap".

[johnj]

Okay, wonderful.

Thanks Gabi and Death for helping me wrap my head around these issues!

[cypherdoc]

you would find this interesting:

https://bitcointalk.org/index.php?topic=3008.msg565787#msg565787

i also ran this by Gavin and he agrees completely with ethotheipi's analysis.  Gavin thinks we have nothing to worry about for 10 yrs and then we can change the system to handle any threats.

[johnj]

you would find this interesting:

https://bitcointalk.org/index.php?topic=3008.msg565787#msg565787

i also ran this by Gavin and he agrees completely with ethotheipi's analysis.  Gavin thinks we have nothing to worry about for 10 yrs and then we can change the system to handle any threats.

Wonderful, thats a whole lot of other info to incorporate, thank you very much!

[cjp]

Check out this page (warning: Dutch language!):
http://www.knmi.nl/cms/nieuws/nieuwsbericht/_rp_column1-1_elementId/1_105587

Basically it says that the Dutch weather forecast agency now has a new supercomputer, which has 58.2 Tflop, which makes it one of the fastest supercomputers in the country.

I made a small calculation of how much income it could make with BTC mining, and my result was approximately 12 EUR / day (at current hash rate and exchange rate).

I am a bit disappointed by the low number. How can it possibly be so low? I can't imagine you can possibly break-even at 12 EUR/day with a computer like that, so apparently many miners manage to be A LOT more efficient.

And I don't really buy the story that it is much more efficient to do highly parallelized computations on GPUs. AFAIK, weather calculations are highly parallelizable too, so if GPUs were so much better, I'd expect the weather forecast computer to use GPUs as well. In fact, I'd expect this computer of them to contain a GPU-style architecture (although I haven't verified it yet).

Maybe the whole ratio between hash/s and flops/s is wrong? I used the value from bitcoinwatch.com, which currently says:
137.91 PetaFLOPS
10.86 Thash/s

[DeathAndTaxes]

1) 58.2 TFlop is pathetically weak for a super computer.  A 5970 graphics card has about 4.6 TFLOPs.  So it is roughly equal to the floating point math calculations of ~12 5970 GPUs (4x that if it is double precision).

2) Super computers aren't optimized for Bitcoin mining.  They have lots of very expensive parts.  Terrabytes of ram, petabytes of storage, redundant backups, high speed interconnects.  All that adds up to hundreds of millions of dollars and produces 0.0 hashes.

3) The largest super computers in the world is 11,280 TFLOPs.  http://i.top500.org/system/177232

4) That ratio is just a guestimate.  Bitcoin mining uses integer math TFLOPS are a measure of floating point math.  Saying the network is 137.91 PFLOPS isn't exactly accurate.  It is 0 PFLOPS however it likely has the computing hardware that combined is roughly equal to ~100 PFLOPS.