Concerns about Ultraprune

[Atlas]

Atlas I don't understand how storing data differently locally changes how this data is verified to be valid?

Also please slow down, or I'll be forced to start clicking the Delete this reply button.

When you change the format of data, couldn't it be altered if an error is made in implementing the new format?
Wouldn't bad data being verified into the blockchain mistakenly be a problem?

[Polvos]

Atlas, in my opinion, you are doing a great job as a bitcoin spirit guardian. I really appreciate the devil advocate rol in your defense of a independent and decentralized blockchain.

You must really be doing things great when some of the people posting here use ad hominem arguments showing a sheep brainless behaviour to your warnings.

I agree with the need of months of hard testing before launching big or potentially dangerous changes in the system.

[hazek]

Atlas I don't understand how storing data differently locally changes how this data is verified to be valid?

Also please slow down, or I'll be forced to start clicking the Delete this reply button.

When you change the format of data, couldn't it be altered if an error is made in implementing the new format?
Wouldn't bad data being verified into the blockchain mistakenly be a problem?

No it couldn't. Because the "mechanism" inside the entire client code that validates the data is separate from the "mechanism" that stores the data. So even if bad data gets stored, next time it's read it wont pass the valid data test by this or any other client in the network.

Therefor how data is stored is not a protocol change as you've advertised it unless they also changed the mechanism for data validation.

[Gabi]

What atlas says make sense, the problem is how he says that... A title saying "might introduce a serious change to the protocol" means nothing. The thread you opened is about BUGS in the new database system, not about the change itself. The change is fine and will greatly boost the blockchain download speed. As for bugs, well sure, the new system must be tested, but then please get a proper title   Right now it seems the problem is the new protocol!

[Atlas]

What atlas says make sense, the problem is how he says that... A title saying "might introduce a serious change to the protocol" means nothing. The thread you opened is about BUGS in the new database system, not about the change itself. The change is fine and will greatly boost the blockchain download speed. As for bugs, well sure, the new system must be tested, but then please get a proper title   Right now it seems the problem is the new protocol!

Yeah, thanks.

Hazel, my concern would then be is if these new data stores can interfere with the verification process unintentionally due to extraneous code.

My knowledge is shallow, that is true. However, I think we can agree the blockchain makes Bitcoin.

[deepceleron]

Welcome to my ignore list. Trolling the other pull thread too in addition to this? Your ignore button can now be even more orange.

Changes to how the blockchain is stored on a local client do not affect the Bitcoin protocol or the way signature verifications work. It can only improve things. We already see the failings of the current BerkleyDB format with its single-purpose huge index file, notably the inevitable unrecoverable corruption that comes from writing and rewriting 2GB files, and we are halfway to the file size limit of the hard-coded index. Many alternate clients already store the blockchain in different ways.

[Atlas]

Welcome to my ignore list. Trolling the other pull thread too in addition to this? Your ignore button can now be even more orange.

Changes to how the blockchain is stored on a local client do not affect the Bitcoin protocol or the way signature verifications work. It can only improve things. We already see the failings of the current BerkleyDB format with its single-purpose huge index file, notably the inevitable unrecoverable corruption that comes from writing and rewriting 2GB files, and we are halfway to the file size limit of the hard-coded index. Many alternate clients already store the blockchain in different ways.

Thanks, things are clearer now but I remain skeptical.

I do not fear being wrong in this case. I only fear being right. Your ignores won't influence me.

[hazek]

What atlas says make sense, the problem is how he says that... A title saying "might introduce a serious change to the protocol" means nothing. The thread you opened is about BUGS in the new database system, not about the change itself. The change is fine and will greatly boost the blockchain download speed. As for bugs, well sure, the new system must be tested, but then please get a proper title   Right now it seems the problem is the new protocol!

Yeah, thanks.

Hazek, my concern would then be is if these new data stores can interfere with the verification process unintentionally due to extraneous code.

It could, but that's a bug, not a protocol change.

My knowledge is shallow, that is true.

You are quite right, it's also the source of all these ad hominem attacks against you from people who do understand it and can easily tell just how wrong you are.

However, I think we can agree the blockchain makes Bitcoin.

It does not. What makes bitcoin are rules that govern it and the mechanisms that are at work that apply these rules. The existence of the blockchain is such a mechanism, the structure of the blockchain is not.

[Atlas]

What atlas says make sense, the problem is how he says that... A title saying "might introduce a serious change to the protocol" means nothing. The thread you opened is about BUGS in the new database system, not about the change itself. The change is fine and will greatly boost the blockchain download speed. As for bugs, well sure, the new system must be tested, but then please get a proper title   Right now it seems the problem is the new protocol!

Yeah, thanks.

Hazek, my concern would then be is if these new data stores can interfere with the verification process unintentionally due to extraneous code.

It could, but that's a bug, not a protocol change.

My knowledge is shallow, that is true.

You are quite right, it's also the source of all these ad hominem attacks against you from people who do understand it and can easily tell just how wrong you are.

However, I think we can agree the blockchain makes Bitcoin.

It does not. What makes bitcoin are rules that govern it and the mechanisms that are at work that apply these rules. The existence of the blockchain is such a mechanism, the structure of the blockchain is not.

What I fear is a bug then which has been my fear from the OP. All I want is this code to be looked over very carefully.

Anyways, it's great to be wrong.

[hazek]

I suggest you change your OP to reflect that.

Nvm, I see you already have..

[Pieter Wuille]

If we change how blocks are verified and mined, and that is done with a bug included, it could lead to massive damage for the Bitcoin network as a whole since verification is everything.

Yes, of course. This is a very fundamental change, and it will require an extraordinary amount of testing before this will end up in a release. No one is thinking about just releasing this right now. This took a long time to write and tweak to get it in its current form, and took all precautions I could to make sure it behaves exactly as the old code - I'm all aware of the risks if it happens to contain a bug. However, in my opinion it is also the right way forward. We need to make sure fully validating nodes can keep running on end-user hardware, without crippling their machines.

How this was tested:

• I've received and sent coins with it.
• I've p2pool-mined on an ultraprune Bitcoind with 1 GH/s for a few weeks. An actual block was found.
• I've written code to calculate a hash of the uncompressed database state, both for the old and new schema's. In every test, they result in exactly the same hash for the same block.

Nonetheless, the change is so large that claiming that it is bug free would be naive (and in fact, a bug was probably found already, as you pointed out). That is why I asked for testers in the announcements - we need the bugs to be found now and not later.

Can you now please change the title of this thread? I find this very offensive. I've invested weeks of my personal time in doing what is necessary to deal with the increasing scalability probles, and I get this in return? I have no problem with being cautious, but there is no need for fear mongering.

[Prattler]

This should not be taken lightly at all. They want to restructure the entire blockchain which could potentially break Bitcoin if implemented poorly and all the nodes accept it.

Push them not to implement this change into Bitcoin-Qt/Bitcoind until it has been tested for at least 6 months.

Atlas and Co is the real DANGER. This change will benefit bitcoin A LOT!

[paraipan]

Atlas, that is the blockchain prune you prick!  

Peter is working for almost a year on it, so you better start learning how to read code and don't spend you energy on stupid threads like this one. In the future "the literate" could be only the people who know how to read and write computer code, the rest will still believe in withes and magic like they always did.

[LightRider]

Technical problems call for technical solutions, not wild arm flailing DANGER WILL ROBINSON screaming. Use testnet, report bugs, be constructive. This thread isn't helping your cause.

[kokjo]

Technical problems call for technical solutions, not wild arm flailing DANGER WILL ROBINSON screaming. Use testnet, report bugs, be constructive. This thread isn't helping your cause.

Atlas is a Computer Noob, can't code, and i don't think he knows how to use a terminal.

[freequant]

Atlas, I've got some reading for you

[Herodes]

https://bitcointalk.org/index.php?topic=119525.0

This should not be taken lightly at all. They want to restructure the entire blockchain which could potentially break Bitcoin if implemented poorly and all the nodes accept it.

Push them not to implement this change into Bitcoin-Qt/Bitcoind until it has been tested for at least 6 months.

I saw the title of the thread, secondly I checked whether or not it was Atlas starting the topic. Once determined it in deed was Atlas, care no more about this thread.

[Stapleddiet]

Atlas you seem to have a good grasp of english, perhaps you could use that to voicing your concerns in a different way. If you are really concerned about how Bitcoin fares, less sensational sounding topics to go to google would be helpful to the whole project overall.

[Yankee (BitInstant)]

While I disagree with your methods of communication and your motives sometimes, I do agree with you here.

As Bitcoin progresses and scales up, much more is at stake.

Not only should we wait a few months and thoroughly tested, but a standard path of Quality Assurance should be implemented.

[MysteryMiner]

As I understand the biggest change is to a higher performance database engine and logical layout of blockchain storage on local node. To ease the mind - some alternate clients like Ufasoft Coin use custom database already that is more compact size and someone might have mined a block using Ufasoft Coin.

Will the new 0.8 client be able to communicate with 0.3.xx properly? If Yes, then it is OK because no protocol is changed. Then please not upgrade all at once and give diversity and time to test it large scale.

Are bugs dangerous? Of course they are, we must have a plan what to do when something breaks. Testing, more testing and code reviews are the way to rule out any bugs.

Wouldn't bad data being verified into the blockchain mistakenly be a problem?

There was a serious bug back in 2010 that caused blockchain rollback. Look up Wiki for this.

Still the ASCII Bernanke would be there forever!