Bitcoin Forum

https://bitcointalk.org/index.php?topic=119525.0

This should not be taken lightly at all. They want to restructure the entire blockchain which could potentially break Bitcoin if implemented poorly and all the nodes accept it.

Push them not to implement this change into Bitcoin-Qt/Bitcoind until it has been tested for at least 6 months.

Additionally, DO NOT ACCEPT a release of this kind until it has been tested for a long period of time regardless of calls for trust. This isn't a kangaroo court. This isn't Zion. This is Bitcoin, the money of a new age.

I have never upgraded without at least a month delay.  Even if it's to deal with a 'critical' bug, I have just turned off my client and don't turn it on until others have had time to test it for me.

I can't see how that proposal would change the blockchain, it looks like it only changes how the blockchain is parsed locally.

Well, I need to reword what I said. The following looks like it might need to change how blocks are verified and mined:

A small summary of the changes:
Instead of blk000?.dat, we have blocks/blk000??.dat files of max 128 MiB, pre-allocated per 16 MiB
Instead of a Berklely DB blkindex.dat, we have a LevelDB directory blktree/. This only contains a block index, no transaction index.
A new LevelDB directory coins/, which contains data about the current unspent transaction output set.
New files blocks/rev000??.dat contain undo data for blocks (necessary for reorganisation).
More information is kept about blocks and block files, to facilitate pruning in the future, and to prepare for a headers-first mode.
Two new RPC calls are added: gettxout and gettxoutsetinfo.

If we change how blocks are verified and mined, and that is done with a bug included, it could lead to massive damage for the Bitcoin network as a whole since verification is everything.

Dude, will you whistleblow for every change the devs introduce from now on ? Bitcoin needs to evolve continuosly, though I'm very conservative and wish to keep their original virtues.

If you are really concerned about future changes, I suggest you either :

1) Learn programming and participate in the code auditing.
2) Pay for a third party auditor to do that job.

You can do #2 thru a "bitcoin auditing task force" and donations or something similar.

It doesn't take a code audit to understand that changing the structure of the blockchain that will then be verified, which sets history for all Bitcoin holders, is a serious change that can go wrong.

"Anything that can go wrong will go wrong".

This is not a protocol change, idiot.

It changes how the network operates and how some will verify the blockchain.

My vocabulary may be off but I stand by my assertion that this could compromise the verification process of transactions.

I care not if the problem would be detected quickly. A problem brings instability to the ecosystem.

I vote for rigid conservation. This is not a experiment.

Jgarzik is already having problems with this new build. His Bitcoins are not showing up under it.

http://bitcoinstats.com/irc/bitcoin-dev/logs/2012/10/21

BUGS ARE IN THIS AS IS

You are unique atlas.

Please, can everyone not just ignore this guy? I mean, really ignore him. Don't even open his posts (I only even bothered myself with this thread so that I could get this off my mind). It's the most blatant example of FUD trolling that I have witnessed so far. Watch me ignore him when/if he starts to reply to me (it won't be difficult to resist, as I'm not opening this thread again).

Simply by engaging with him (talking to him, opening his post such that it gets 1 more view count etc) makes it easier for him to do his "job". Make it difficult for him. Stop.

I have no incentive to troll. I have a great incentive to protect Bitcoin.

What the hell is wrong with you, a couple weeks ago you were calling the developers incompetent because the initial block download is not faster. Then it's made faster (on the order  of 6-40x faster depending on the user's hardware, as a result of months of work) and you start screaming about "restructure the entire blockchain and protocol"— neither of which are changed by this improvement. And you can't seem to avoid invoking Gavin, though his only role in this particular set of changes has been reviewing code.

Things were much better on the forum when you and your crazy foaming were banned.

I think even Satoshi himself would disagree with you on that assertion.

When you change the file format of the blockchains files which are then used for transactions, an error in implementing said format change could lead to problems.

I am open to being wrong but the way I see it, this is significant.

110 million dollars doesn't think so.

Atlas I don't understand how storing data differently locally changes how this data is verified to be valid?

Also please slow down, or I'll be forced to start clicking the Delete this reply button.

then the people owning 110m$ of bitcoin, must take responsibility and make THEIR OWN client.

When you change the format of data, couldn't it be altered if an error is made in implementing the new format?
Wouldn't bad data being verified into the blockchain mistakenly be a problem?

Atlas, in my opinion, you are doing a great job as a bitcoin spirit guardian. I really appreciate the devil advocate rol in your defense of a independent and decentralized blockchain.

You must really be doing things great when some of the people posting here use ad hominem arguments showing a sheep brainless behaviour to your warnings.

I agree with the need of months of hard testing before launching big or potentially dangerous changes in the system.

No it couldn't. Because the "mechanism" inside the entire client code that validates the data is separate from the "mechanism" that stores the data. So even if bad data gets stored, next time it's read it wont pass the valid data test by this or any other client in the network.

Therefor how data is stored is not a protocol change as you've advertised it unless they also changed the mechanism for data validation.

What atlas says make sense, the problem is how he says that... A title saying "might introduce a serious change to the protocol" means nothing. The thread you opened is about BUGS in the new database system, not about the change itself. The change is fine and will greatly boost the blockchain download speed. As for bugs, well sure, the new system must be tested, but then please get a proper title  :D Right now it seems the problem is the new protocol!

Yeah, thanks.

Hazel, my concern would then be is if these new data stores can interfere with the verification process unintentionally due to extraneous code.

My knowledge is shallow, that is true. However, I think we can agree the blockchain makes Bitcoin.

Welcome to my ignore list. Trolling the other pull thread too in addition to this? Your ignore button can now be even more orange.

Changes to how the blockchain is stored on a local client do not affect the Bitcoin protocol or the way signature verifications work. It can only improve things. We already see the failings of the current BerkleyDB format with its single-purpose huge index file, notably the inevitable unrecoverable corruption that comes from writing and rewriting 2GB files, and we are halfway to the file size limit of the hard-coded index. Many alternate clients (https://en.bitcoin.it/wiki/Category:Clients) already store the blockchain in different ways.

Thanks, things are clearer now but I remain skeptical.

I do not fear being wrong in this case. I only fear being right. Your ignores won't influence me.

It could, but that's a bug, not a protocol change.


You are quite right, it's also the source of all these ad hominem attacks against you from people who do understand it and can easily tell just how wrong you are.


It does not. What makes bitcoin are rules that govern it and the mechanisms that are at work that apply these rules. The existence of the blockchain is such a mechanism, the structure of the blockchain is not.

What I fear is a bug then which has been my fear from the OP. All I want is this code to be looked over very carefully.

Anyways, it's great to be wrong.

I suggest you change your OP to reflect that.

Nvm, I see you already have..

Yes, of course. This is a very fundamental change, and it will require an extraordinary amount of testing before this will end up in a release. No one is thinking about just releasing this right now. This took a long time to write and tweak to get it in its current form, and took all precautions I could to make sure it behaves exactly as the old code - I'm all aware of the risks if it happens to contain a bug. However, in my opinion it is also the right way forward. We need to make sure fully validating nodes can keep running on end-user hardware, without crippling their machines.

How this was tested:

• I've received and sent coins with it.
• I've p2pool-mined on an ultraprune Bitcoind with 1 GH/s for a few weeks. An actual block was found.
• I've written code to calculate a hash of the uncompressed database state, both for the old and new schema's. In every test, they result in exactly the same hash for the same block.

Nonetheless, the change is so large that claiming that it is bug free would be naive (and in fact, a bug was probably found already, as you pointed out). That is why I asked for testers in the announcements - we need the bugs to be found now and not later.

Can you now please change the title of this thread? I find this very offensive. I've invested weeks of my personal time in doing what is necessary to deal with the increasing scalability probles, and I get this in return? I have no problem with being cautious, but there is no need for fear mongering.

Atlas and Co is the real DANGER. This change will benefit bitcoin A LOT!

Atlas, that is the blockchain prune you prick!  :-\

Peter is working for almost a year on it, so you better start learning how to read code and don't spend you energy on stupid threads like this one. In the future "the literate" could be only the people who know how to read and write computer code, the rest will still believe in withes and magic like they always did.

Technical problems call for technical solutions, not wild arm flailing DANGER WILL ROBINSON screaming. Use testnet, report bugs, be constructive. This thread isn't helping your cause.

Atlas is a Computer Noob, can't code, and i don't think he knows how to use a terminal.

Atlas, I've got some reading for you (http://en.wikipedia.org/wiki/The_Boy_Who_Cried_Wolf)

I saw the title of the thread, secondly I checked whether or not it was Atlas starting the topic. Once determined it in deed was Atlas, care no more about this thread.

Atlas you seem to have a good grasp of english, perhaps you could use that to voicing your concerns in a different way. If you are really concerned about how Bitcoin fares, less sensational sounding topics to go to google would be helpful to the whole project overall.

While I disagree with your methods of communication and your motives sometimes, I do agree with you here.

As Bitcoin progresses and scales up, much more is at stake.

Not only should we wait a few months and thoroughly tested, but a standard path of Quality Assurance should be implemented.

As I understand the biggest change is to a higher performance database engine and logical layout of blockchain storage on local node. To ease the mind - some alternate clients like Ufasoft Coin use custom database already that is more compact size and someone might have mined a block using Ufasoft Coin.

Will the new 0.8 client be able to communicate with 0.3.xx properly? If Yes, then it is OK because no protocol is changed. Then please not upgrade all at once and give diversity and time to test it large scale.

Are bugs dangerous? Of course they are, we must have a plan what to do when something breaks. Testing, more testing and code reviews are the way to rule out any bugs. There was a serious bug back in 2010 that caused blockchain rollback. Look up Wiki for this.

Still the ASCII Bernanke would be there forever!

Yes.

From the P2P side of things, you can't even observe to what kind of node you are talking (except for the version number being announced, and assuming no bugs that give it away).

Then the "blockchain reordering" and "protocol change" concerns of Atlas are void.

One more question - will the new database discard spent addresses? Some places says it will, some says it will not. I am confused. What will happen to clients that rely on downloading the complete transaction history and verify all blocks and transactions in them on-the-way, like 0.3.xx does?

Atlas, weren't you bitching earlier about the necessity to download the whole blockchain on every node?

Now something is done about that and you hit on that?  ???
How about it: Convince some Alternate Cyptocurrency devs to test it out before bitcoin does it if you really are that concerned... Do something productive for gods sake.

Please just read the thread in the dev forum about it.

I think it was this one https://bitcointalk.org/index.php?topic=119525.0

By all blocks this means all full block contents or headers of all blocks?So the validation code but not validation rules is changed? Atlas concern about hidden bug might hold some water then. Also how will the new validating client behave when validating double-spend when the double-spends are with large time spread like few months?

Why are you using the pronoun WE? What part of the Bitcoin code have you contributed?

As I told you: the behaviour of a new node and an old node should be indistinguishable, from the P2P side.

This means it will create the same blocks. It will accept the same blocks, and ignore the same invalid blocks. It will relay the same blocks and transactions. It will follow exactly the same rules. You cannot tell their behavior apart, except that it is faster.

The only difference is a) the database technology changed and b) we use a set of unspent transaction outputs instead of an index into the block chain that holds the same data. A transaction output is removed from the set as soon as it is spent, and only then. This means that even if they are a hundred years apart, a double spend will not find the inputs it wants to consume, and fail.

This is not a change of the block chain. This is an evolution in the technology for validating it, and be sure that it will be tested thoroughly.

... you're feeding it  :)

This.

Atlas you have good intentions and and enviable level of energy, but need to pick your battles, consult with others, and force yourself to take a timeout (my suggestion: 24hs) before starting panic threads with incendiary accusations and allegations.  There is nothing nefarious, and much to be happy about, wrt. the arrival of client blockchain handling optimizations.  The developers, wishing to avoid crowds with pitchforks at their door, will have every incentive to ensure a long test cycle that preserves the integrity of the network as well as everyone's existing wallet.  

It would be a career-ending move to introduce bugs that lost people's money or allowed invalid transactions to be accepted.  We all (including you) should be grateful that Pieter is willing to do this work for free, and chosen to put on his shoulders a great deal of risk and responsibility in order to make the reference client a more usable product.

Also, let's discuss the technology itself on the development thread (https://bitcointalk.org/index.php?topic=119525.0), not here.

Yes, I suppose you are right. But with regards to Atlas I suppose this describes it (http://www.youtube.com/watch?v=fL6wbsGx9qw).

No. And if u r not smart enough to get why ur advice is bad then u'd better just leave this site, coz its content is beyond ur comprehension.

Everyone who runs a Bitcoin node contributes. You don't need to write a code to contribute in a P2P system.This is answer that can put a lock on this thread. Change in database engine or structure is not going to change the results produced. It is like going from blockchain stored in plaintext databases to MySQL databases. All it can change is a performance. This is how far my knowledge in databases goes.

P.S. I also did not know that 0.3.xx have a hardcoded 4GB database limit. This probably means my 0.3.xx clients will stop functioning sometime in future. Is this explicitly written in code or something to do with BerkleyDB database engine or 32-bit limitation?

There is no room to be offended at skepticism when: 1) You guys are claiming an official release through Bitcoin.org and 2) when you're dealing with a highly valued asset.

This isn't a pet project. As much as Gavin Andresen insists that nobody should take Bitcoin seriously and that he should fuck up as he likes with no liability, I say nay. I say we should treat Bitcoin as the revolutionary tool it ought to be. 

Your concerns have been duely noted, Atlas.  However, you have little knowledge of what is really going on here, so I consider your perspectives discounted.  Just don't upgrade yourself, and if others do and are harmed by it, you will be able to say "I told you so" because you have.

But the personal attacks will end.  I don't know what problems you have with Gavin and some of the other developers, but if you want a soapbox to attack the developers you are going to have to find somewhere else to do it.

No personal attacks have been made against their persons but only their actions in the Bitcoin ecosystem. This should be encouraged.

To say I have personal issues with certain individuals is libellous.

All people with power in this ecosystem shall be questioned.