Data rot: how does bitcoin handle it?

[TonyT]

http://en.wikipedia.org/wiki/Data_degradation ("Data degradation in memory can occur when the small electric charge of a bit in RAM disperses, possibly altering program code or stored data. The hypothesis that semiconductor RAM may occasionally be altered by cosmic rays[2] is also known as soft error.")

A while ago --within the last two years I think-- Google claimed that one bit in a server farm rotted (I vaguely recall this is a 1 in a 10^12 event, but if you have enough bits it's possible), and this error somehow propagated and crashed their entire network.  Given this was Google who hires the best talent money can bribe, I wonder what would happen if somehow the BTC blockchain, if there is a canonical or gold standard version somewhere, suffered data rot, let's say a solar flare produced lots of cosmic particles that caused an error?  Would the entire BTC ecosystem come to a screeching halt?  If it happened to Google, it can happen here? 

[Buffer Overflow]

The network is distributed across thousands of machines. An error in one wouldn't effect the remaining nodes.

[katlogic]

http://en.wikipedia.org/wiki/Data_degradation ("Data degradation in memory can occur when the small electric charge of a bit in RAM disperses, possibly altering program code or stored data. The hypothesis that semiconductor RAM may occasionally be altered by cosmic rays[2] is also known as soft error.")

Private keys can be subject to bit flip errors, so individual wallet can lose coins if there is no backup to restore the wallet from. In that case it is usually also possible to recover the key by trying to undo the bitrot in bruteforce manner, usually the search space is rather small.

As for the rest, the network is embarrassingly resilient (faulty node is no different from evil node).

[TonyT]

http://en.wikipedia.org/wiki/Data_degradation ("Data degradation in memory can occur when the small electric charge of a bit in RAM disperses, possibly altering program code or stored data. The hypothesis that semiconductor RAM may occasionally be altered by cosmic rays[2] is also known as soft error.")

Private keys can be subject to bit flip errors, so individual wallet can lose coins if there is no backup to restore the wallet from. As for the rest, the network is embarrassingly resilient (faulty node is no different from evil node).

Thanks for the answer.  Well that's nice, but with Google, they did have RAID and apparently that was not good enough.  Let me find the article... no, I could not, but LOL I actually see this thread is already indexed by Google, so let's stay on topic and be nice so posterity can benefit from reading this...  the best I could find is here: http://serverfault.com/questions/77710/is-bit-rot-on-hard-drives-a-real-problem-what-can-be-done-about-it

So in a RAID system, where there's all kinds of redundancy, you can have bit rot that creates problems. So why not also in the Bitcoin peer-to-peer network?  I think it's possible, if somebody says: "Adopt this blockchain, it is the best one" and they are trustworthy, and people adopt it, but the blockchain has bit rot?  Anyway I'm not going to worry about it, it's not my job.

[shorena]

-snip-
"Adopt this blockchain, it is the best one" and they are trustworthy, and people adopt it, but the blockchain has bit rot?  Anyway I'm not going to worry about it, it's not my job.

Thats not how bitcoin works. No full node just trusts another node. Most of your threads I read in the recent past could be avoided if you read the whitepaper.

[TonyT]

-snip-
"Adopt this blockchain, it is the best one" and they are trustworthy, and people adopt it, but the blockchain has bit rot?  Anyway I'm not going to worry about it, it's not my job.

Thats not how bitcoin works. No full node just trusts another node. Most of your threads I read in the recent past could be avoided if you read the whitepaper.

You make an ad hominem argument, in an attempt to sound authoritative ("most of your threads"), a common rhetorical tactic.

Educate yourself and get back to us (note the royal plural) by reading this thread, pay attention to the quoted language:

https://github.com/bitcoin/bips/blob/master/bip-0050.mediawiki     ("Marek Palatinus and Michael Marsee quickly downgraded their nodes to restore a pre-0.8 chain as canonical, despite the fact that this caused them to sacrifice significant amounts of money and they were the ones running the bug-free version.")

TonyT

[Buffer Overflow]

All nodes on the network are equal and automatically assume 100% that information received from connected peers is malicious until independently verified by itself.
This is the cornerstone security model of the bitcoin network.

Any data received from a peer node containing invalid data (deliberate or by accident) will be ignored.

[shorena]

-snip-

You make an ad hominem argument,

Yes I criticised you as a person or rather your behaviour when opening threads.

in an attempt to sound authoritative ("most of your threads"), a common rhetorical tactic.

No, I was just hinting you at the solution. I have no intention to sound "authorative". It was just the opservation I made in the last days reading your threads.

[TonyT]

All nodes on the network are equal and automatically assume 100% that information received from connected peers is malicious until independently verified by itself.
This is the cornerstone security model of the bitcoin network.

Any data received from a peer node containing invalid data (deliberate or by accident) will be ignored.

Yes, I agree, most of the time this is true.  But see this post below and consider the implications if the "canonical" blockchain had bit rot during this critical moment.

TonyT

https://github.com/bitcoin/bips/blob/master/bip-0050.mediawiki     ("Marek Palatinus and Michael Marsee quickly downgraded their nodes to restore a pre-0.8 chain as canonical, despite the fact that this caused them to sacrifice significant amounts of money and they were the ones running the bug-free version.")

[Buffer Overflow]

All nodes on the network are equal and automatically assume 100% that information received from connected peers is malicious until independently verified by itself.
This is the cornerstone security model of the bitcoin network.

Any data received from a peer node containing invalid data (deliberate or by accident) will be ignored.

Yes, I agree, most of the time this is true.  But see this post below and consider the implications if the "canonical" blockchain had bit rot during this critical moment.

TonyT

https://github.com/bitcoin/bips/blob/master/bip-0050.mediawiki     ("Marek Palatinus and Michael Marsee quickly downgraded their nodes to restore a pre-0.8 chain as canonical, despite the fact that this caused them to sacrifice significant amounts of money and they were the ones running the bug-free version.")

This was caused by the infamous upgrade bug.

You can't have bit rot on the blockchain, unless every node had the same bit rot. If a node had bit rot, the rest of the non-bit rotted nodes would simply ignore it and carry on as usual.

[Rannasha]

The fork that happened with the update to 0.8 was of a completely different nature than what would happen if there is bit-rot in a node. The 0.8 fork was due to different nodes using different sets of rules regarding which blocks to accept as valid. So 0.8 nodes would accept blocks that the rest of the network would not.

In the event of bit-rot, a block simply fails validation. All transactions in a block affect the merkle root in the block header, so a change in transactions changes the block header. And the block header, including the nonce, is hashed to obtain a value below the target (related to the mining difficulty) in order for a block to be valid. Flip a single bit and this validation will fail. Other nodes will reject the block and the affected node will eventually replace the corrupted block by its uncorrupted version.

[TonyT]

The fork that happened with the update to 0.8 was of a completely different nature than what would happen if there is bit-rot in a node. The 0.8 fork was due to different nodes using different sets of rules regarding which blocks to accept as valid. So 0.8 nodes would accept blocks that the rest of the network would not.

In the event of bit-rot, a block simply fails validation. All transactions in a block affect the merkle root in the block header, so a change in transactions changes the block header. And the block header, including the nonce, is hashed to obtain a value below the target (related to the mining difficulty) in order for a block to be valid. Flip a single bit and this validation will fail. Other nodes will reject the block and the affected node will eventually replace the corrupted block by its uncorrupted version.

Fine and good, thanks, but consider this:  there was a 'canonical' pre-0.8 chain.  Do a 'thought experiment':  what happened if this pre-0.8 chain had bit rot?  And Palatinus and Marsee introduced it?  It would be rejected?  Then the blockchain in March 2013 would have forked, causing double spends and ruining Bitcoin forever?  Apparently, if I read Gavin Andresen's memo correctly, only the presence of a 'canonical' pre-0.8 chain saved bitcoin.  So if this 'canonical' chain was rotted, then what?  Maybe there was another chain?  Or something else?

TonyT

https://github.com/bitcoin/bips/blob/master/bip-0050.mediawiki     ("Marek Palatinus and Michael Marsee quickly downgraded their nodes to restore a pre-0.8 chain as canonical, despite the fact that this caused them to sacrifice significant amounts of money and they were the ones running the bug-free version.")

[Syke]

Fine and good, thanks, but consider this:  there was a 'canonical' pre-0.8 chain.  Do a 'thought experiment':  what happened if this pre-0.8 chain had bit rot?

There are something like 10,000 copies of the chain. It would be impossible for all copies to have the same bit rot.

[TonyT]

Fine and good, thanks, but consider this:  there was a 'canonical' pre-0.8 chain.  Do a 'thought experiment':  what happened if this pre-0.8 chain had bit rot?

There are something like 10,000 copies of the chain. It would be impossible for all copies to have the same bit rot.

Unless I'm reading it wrong, it's not necessary for all copies to have bit rot, just the canonical copy that either Palatinus or Marsee had.  Perhaps you are correct if by 'downgrading' it means the existing nodes out there--the 10000 copies you mention--would, by nature of the P2P network, becomes the majority again, but it's not clear since the mere fact that Palatinus and Marsee uploaded their nodes with a post-0.8 chain made the chain unstable.  So it implies that two people--Palatinus and Marsee--had control of the entire bitcoin network by virtue of having a canonical node(s).  Thus if these two nodes had bit rot, they would break the system.  That's the implication.  And I speculate that's how in fact Bitcoin p2P works:  the people who run nodes 'know' that Palatinus and Marsee have the 'good stuff' and trust them, when there is an upgrade, to download and install the blockchain these two folk (and others trusted like them) provide.

TonyT

https://github.com/bitcoin/bips/blob/master/bip-0050.mediawiki     ("Marek Palatinus and Michael Marsee quickly downgraded their nodes to restore a pre-0.8 chain as canonical, despite the fact that this caused them to sacrifice significant amounts of money and they were the ones running the bug-free version.")

[Buffer Overflow]

Fine and good, thanks, but consider this:  there was a 'canonical' pre-0.8 chain.  Do a 'thought experiment':  what happened if this pre-0.8 chain had bit rot?

There are something like 10,000 copies of the chain. It would be impossible for all copies to have the same bit rot.

It's not necessary for all copies to have bit rot, just the canonical copy that either Palatinus or Marsee had.

TonyT

As I said, the rest of the un-rotted network would see the very obvious rot and ignore it, and carry on as normal.

[TonyT]

Fine and good, thanks, but consider this:  there was a 'canonical' pre-0.8 chain.  Do a 'thought experiment':  what happened if this pre-0.8 chain had bit rot?

There are something like 10,000 copies of the chain. It would be impossible for all copies to have the same bit rot.

It's not necessary for all copies to have bit rot, just the canonical copy that either Palatinus or Marsee had.

TonyT

As I said, the rest of the un-rotted network would see the very obvious rot and ignore it, and carry on as normal.

But if bit rot can bring down a RAID system, which has redundancy built in, why do you posit it cannot do the same with the bitcoin blockchain?  Something different about how the bitcoin blockchain is signed?  Perhaps this is true, since with RAID bit rot, apparently the bit rotted image of the RAID HD is indistinguishable from the non-bit rotted image (not clear to me why, with all the parity checks in RAID, but that's what I recall reading), whereas with a P2P network you have more than one 'image' to deal with and more than just parity checks.  However, even if true, this brings up another issue: if Palatinus or Marsee tell the Bitcoin nodes:  'we are having a hard fork, use this transaction block from now on' and they upload it, and it is adopted, and it has bit rot, then what?  Or is this against the Bitcoin rules?

[Syke]

Unless I'm reading it wrong, it's not necessary for all copies to have bit rot, just the canonical copy that either Palatinus or Marsee had.  Perhaps you are correct if by 'downgrading' it means the existing nodes out there--the 10000 copies you mention--would, by nature of the P2P network, becomes the majority again, but it's not clear since the mere fact that Palatinus and Marsee uploaded their nodes with a post-0.8 chain made the chain unstable.  So it implies that two people--Palatinus and Marsee--had control of the entire bitcoin network by virtue of having a canonical node(s).  Thus if these two nodes had bit rot, they would break the system.

No, because that's where the mining process comes in. Every block is validated with the hashes. Any "rot" would invalidate one person's chain and someone else's copy of the chain would be used.

[TonyT]

Unless I'm reading it wrong, it's not necessary for all copies to have bit rot, just the canonical copy that either Palatinus or Marsee had.  Perhaps you are correct if by 'downgrading' it means the existing nodes out there--the 10000 copies you mention--would, by nature of the P2P network, becomes the majority again, but it's not clear since the mere fact that Palatinus and Marsee uploaded their nodes with a post-0.8 chain made the chain unstable.  So it implies that two people--Palatinus and Marsee--had control of the entire bitcoin network by virtue of having a canonical node(s).  Thus if these two nodes had bit rot, they would break the system.

No, because that's where the mining process comes in. Every block is validated with the hashes. Any "rot" would invalidate one person's chain and someone else's copy of the chain would be used.

Perhaps another way of looking at this problem is the gitbhub thread below.  If, as the chief scientist  G.A. claims, it was only due to the heroic efforts of two people to downgrade to a earlier version of the BTC blockchain that saved the bitcoin P2P network from disaster, then it stands to reason that bitrot could cause a repeat of this incident, in that a 'buggy' version of the bitchain is adopted (namely, one with bitrot) and if there's no heroic people to step in, the system crashes.

Another way of looking at it:  RAID systems have on rare occasions had bit rot that propagates, bringing down the whole system.  In theory this is impossible, but in practice it is not.  Might the same thing happen with bitcoin?  Time will tell.  I'll leave the last word to somebody else.

TonyT

https://github.com/bitcoin/bips/blob/master/bip-0050.mediawiki     ("Marek Palatinus and Michael Marsee quickly downgraded their nodes to restore a pre-0.8 chain as canonical, despite the fact that this caused them to sacrifice significant amounts of money and they were the ones running the bug-free version.")

[spin]

If you don't believe what people are saying, why don't you try it yourself?  

You can see the impact of bitrot yourself.  Open one of your blocks files (blk files) and change a bit, heck, change a byte, or even delete a block file.

I'd take a bet that your bitcoin will keep running until it needs to serve that particular bit, byte or block from that particular file in which case it might crash.  

The node that requested that data from yours will not recieve it, decide to ask the next node and carry one.  If somehow it recieves invalid data it will check it (as it does with all data from other nodes) and discard it.  If it kept recieving invalid data from your node it will disconnect from your node.

The network won't notice anything.

If you corrupt your wallet file, you will notice it (missing bitcoin) but the network won't notice that either.

[hhanh00]

Unless I'm reading it wrong, it's not necessary for all copies to have bit rot, just the canonical copy that either Palatinus or Marsee had.  Perhaps you are correct if by 'downgrading' it means the existing nodes out there--the 10000 copies you mention--would, by nature of the P2P network, becomes the majority again, but it's not clear since the mere fact that Palatinus and Marsee uploaded their nodes with a post-0.8 chain made the chain unstable.  So it implies that two people--Palatinus and Marsee--had control of the entire bitcoin network by virtue of having a canonical node(s).  Thus if these two nodes had bit rot, they would break the system.

No, because that's where the mining process comes in. Every block is validated with the hashes. Any "rot" would invalidate one person's chain and someone else's copy of the chain would be used.

Perhaps another way of looking at this problem is the gitbhub thread below.  If, as the chief scientist  G.A. claims, it was only due to the heroic efforts of two people to downgrade to a earlier version of the BTC blockchain that saved the bitcoin P2P network from disaster, then it stands to reason that bitrot could cause a repeat of this incident, in that a 'buggy' version of the bitchain is adopted (namely, one with bitrot) and if there's no heroic people to step in, the system crashes.

Another way of looking at it:  RAID systems have on rare occasions had bit rot that propagates, bringing down the whole system.  In theory this is impossible, but in practice it is not.  Might the same thing happen with bitcoin?  Time will tell.  I'll leave the last word to somebody else.

TonyT

The blockchain has anti-tempering code embedded inside. It protects it from easy modification. One has to spend a significant computing power in order to add a block. This catches any sort of modification: whether voluntarily or accidental. A bit rot is a random change - it will be detected and rejected.

The event you mentioned is of a different nature. Half of nodes were running an incompatible version of the software and were actively pushing a blockchain that the other half would reject.

In one case, it's an isolated incident that can be caught by the hash. In the other case, it's a widespread software bug that affects a large number of nodes - as if they all went crazy.

We don't have to worry about bit-rot. On the other hand software bugs can be a real threat.

https://github.com/bitcoin/bips/blob/master/bip-0050.mediawiki     ("Marek Palatinus and Michael Marsee quickly downgraded their nodes to restore a pre-0.8 chain as canonical, despite the fact that this caused them to sacrifice significant amounts of money and they were the ones running the bug-free version.")

These guys run mining pools. They represented a big portion of the hashing power at the time. They were running the good version but downgraded to the bad one so that the majority would be clear. In any case, it is not a single machine.

PS: RAID drives are not as well protected as the blockchain. It's a trade off between performance, space and cost. They could detect bit-rot better but they would run slower.