Bitcoin Forum

Protocols often include design decisions that people later regret, but often are kept for the sake of backwards compatibility. Does anyone consider any aspects of the Bitcoin protocol this way? If you could rewrite the protocol now, what things would you change?

The absolute #1 first thing I would change would be to make the network protocol big endian.

Is there any reason in particular? Endianness is pretty straightforward to convert, so I don't see why it would matter either way. Additionally, why wouldn't you use little endian, to match the endianness of x86-based OSes?

Because it goes over the network. I care not what's on the local machine, when dealing with the network. (There's a reason they call big endian "network byte order.")

And x86-based OSes are irrelevant for several deployment scenarios, such as Java (which is big endian).

As for endian conversion, we already have libraries for converting network byte order (big endian) to host byte order (big or little endian; whatever the local system supports), making endian conversion (if necessary) trivial, but only when the protocol is big endian.

I would argue it's trivial either way, with or without a library. It should only take a few lines of code to perform the appropriate bit shifts.

What about security-related things. I heard there is no encryption between directly-connected clients, so that seems like something that would be worth changing.

Fixed little endian is just fine, and happens to match 99.9% of our current usage.

Big endian adds pointless byteswapping.  "network endian" is just so that Sun could sell more hardware, back in the day.

Eh? Sun barely existed on January 1, 1983.

Would use INT128 so it can be claimed that Bitcoin is 'infinitely divisible' (well that's near enough for most people) as a selling point.

Right now saying it's 8 decimals divisible isn't so catchy.

Why not just store numbers in packets as ASCII characters? That would eliminate both length restrictions as well as the above-mentioned endianness concerns. Bittorrent's bencode stores them this way for this reason.

Shh! Don't tell anybody that plain text is big endian! :D :D :D

ASCII is a single-byte encoding scheme so its characters are represented in memory the exact same way on big endian and little endian systems.

Unfortunately this thread seems to be devolving so if anyone else has other aspects of the protocol they would change, I am interested in hearing about it.

Most of the needed changes can be made in a backwards compatible manner. The subver field in version is currently unused, I'd like to maybe repurpose that at some point into a User-Agent equivalent.

I would change this: http://bitcointalk.org/index.php?topic=1865.0

That combined with the bitcoin supply limit being 1 since it's so sexy and natural.

Little Endian is a pain, and I've said it often enough.

Then the protocol version should evolve independently of the client version. Consolidate some stuff, like "do we really need size fields that can have a length of UINT64?". It's just a cheap trick to optimize the length of the messages, but each message has a fixed header field for the command that is never used fully, why start hacking exceptions in?

And then finally comes my argument for structuring the network topology, in order to scale better and detect partitions (think dynamic degree hypercube).

Edit: nearly forgot my old capabilities proposal: http://bitcointalk.org/index.php?topic=894.0

The blockchain is basically an unalterable public register, and I would suggest to use a forward compatible format which allows storage of possibly arbitrary data, of course at the discretion of the block generating node, for possible fees and up to some maximum blocksize.
My main idea for additional usage would be credentials, which would greatly improve some interactions, going so far as to enable credit.
This needs the host and/or blockchain to be used for blind-signing: blind signs-proc.pdf (http://www.cs.umd.edu/~jkatz/papers/blind_sigs-proc.pdf).
The scripting language is already designed to be very flexible, maybe it would need a few extensions.

Disclaimer: I'm regularly programming in C++, but have not much experience with cryptography, so the details of above paper are a little over my head.

My pet peeve, related to endianness, is treating hashes as numbers and reversing them. A hash is a byte string with a well defined order and it makes no more sense to reverse it than to reverse text.

I would also have used a more common elliptic curve for the crypto, and made some changes to speed it up.

Though this will probably forever be impossible to change, I wonder about why the interval time was set at ten minutes. Was this number just plucked out of the air? I know there is a relationship between smaller interval times and larger number of chain forks, but it seems to me that this number should be as low as possible while maintaining an "acceptable" number of forks. Would five minutes work? two? one?

10 mins was set with propagation times in very large networks in mind. I don't think it's worth changing that. There are only two times that matter - instant and not instant. 10 mins vs 5 isn't going to change much.

It would have to be much longer than 10 minutes to work properly on the Interplanetary Internet. ;D

If you just mean to make it take longer to generate, why can't the protocol just gradually increment the number of iterations of SHA256 you must perform?

If you mean to make it less likely for hash vulnerabilities to be found, I think it may be better to just specify the hash algorithm that is used, so the network can easily move to a new algorithm if SHA256 proves to be insecure (as it eventually will be).

EDIT: The reason I think it may not be a good idea to use such a large variety of hash algorithms is that it would make it harder to create an implementation of bitcoin. If I want to write a bitcoin client in javascript, I can easily find a JS library with SHA256 -- but finding one with whirlpool, RIPEMD320, etc, may prove difficult.

Nice one.
As much as I like it for config files, I think JSON is more straidforward when it comes to parsing.


And this one is beautifully interesting. Why not put any strong hashes in existence in there. Even if one fails totally, we're still good. It would take also longer time before mining centralization, on the long run however it would end up with ASICs too probably. It takes maybe some simplicity out of design and makes implementation more complex, but it works. I like this idea very much.

What I would change (mentioned earlier already), I would lower expected time between blocks found. It seems to me that BTC design takes into account growing computational power, but ignores the fact that network is speeding up too. I really don't think that with current connection speeds 10 minutes is needed for propagation.

The protocol could be more permissive about what kind of key could be used.  Basically it could accept not just elliptic curves, but any openssl supported key.

My grumbles (don't read this as aggressive, I love bitcoin, but you did ask):

• Big endian would have been nicer, just to keep with the tradition set by most other TCP/IP protocols ntohs(), et al sort out the swapping for us anyway.  The Internet Protocol spec requires it for all packet headers, and many other protocols.
• Even if not big endian; it would have been nice to be consistent.  There are big and little endian values in the bitcoin protocol.  Making it necessary to check the protocol documentation to be sure, instead of just remembering one.
• Putting a locally-determined address and port number in an application-level message was a bad idea.  Local applications don't know what their globally visible address is -- they can be behind NAT.  It's not like the remote peer even needs it -- that information is handed over by the operating system when connection is established.  FTP has demonstrated for years what a nightmare this is.  The official client has to jump through "whatsmyip" hoops to get this information, and it was completely unnecessary.  If we're talking single points of failure, as an attacker I'd go for whatsmyip (et al) and bitcoin is in trouble.
• The server half of a connection shouldn't speak first.  It makes life easy for sniffers.
• Is verack even necessary?  Connect.  Client says "I speak version 10".  If the server is willing to speak "version 10" it can answer "I will speak version 10" (regardless of its true version); if it is not then it can say "I will speak version 5", the client can decide if it is willing to speak version 5 and continue or hang up.  verack isn't necessary and makes start up more complicated than needed.
• Did we really need RIPE-MD and SHA256?  If you want 160 bytes of hash, then just truncate the SHA-256 hash.  They are meant to be evenly distributed, so there shouldn't be any grouping issues.
• Why use double SHA-256 for the message checksum?  Checksums are there to ensure that data isn't unintentionally corrupted from A to B.  That checksum doesn't need to be cryptographically secure.  Even if it were -- what advantage is there to double hashing for a checksum?
• Block download should have been most recent first.  Each peer must know what its current block chain tips are.  Those should be requestable by a command.  Then getblocks should have sent the one requested and then its parent, then its parent, etc, etc, until we hit the genesis block.  For comparison, see how git stores its "branches" -- the branch is simply a pointer to the commit at the head of that branch.
• No thought seems to have been given to a single multi-user system -- two users of one computer with two wallets that must each remain private from the other.  The main bitcoin server should be started up during boot, and it should be responsible for making itself identical to any other arbitrary node and the information it holds is just as public.  It would be advantageous to run a local copy simple for speed (and generation). Then, a wallet/transaction client would connect to that (or any other node) to perform transactions.  There are no commands to support such a light client -- e.g. requesting transaction validation/confirmation.
• There is no way to query a node for the transactions it has queued.  If I've been disconnected for a long time, I can use getblocks to find out what I've missed.  There is no way to find out what transaction broadcasts I've missed.  This is relevant for early confirmation indications.  Instead of waiting ten minutes for confirmation, I can at least get a hint that the transaction is queued by my peers.
• 64-bit maximum resolution for VarInt type storage?  The message header will only allow you to send 32-bits of payload; making it impossible that any particular vector in a message will ever need 64-bits to specify its length.  On the same theme: the messages are limited to sizes that are way smaller than 32-bit lengths.
• Considering how freely 64-bit fields have been chucked around, the time fields are 32 bits in some places.  64-bit times are useful.
• ... but not for the version message... which uses 64 bits to store the number of seconds since the unix epoch.  Why?  That's 584 billion years of resolution.  Why not have specified it as number of microseconds past the epoch, in case the extra accuracy is ever useful?  That'll still get you 584 thousand years.
• Why break the version field on base-10 boundaries?  Base-16 boundaries make everything easier to process (not needing division for display) and read in a hex dump.
• Why specify transaction values in "tens of nano coins"?  Again: a base-2 split would be better.  A fixed-point base-2 unit doesn't suffer strange rounding errors from 2-to-10 conversion.  I haven't looked but I'll bet the bitcoin client is filled with loads of "convertNanoCoinsToCoins64BitWithRounding()" type functions.  Those special rounding rules have to be implemented by all future clients now.
• Why is there no field in the version message for specifying the application version?  Application and protocol versions are different things.  As more implementations of the client appear, this is important.  What if version 5.6 of SomeNewClient is widely distributed but has a bug in it?  That bug could be worked around if only we knew that we were talking to version 5.6 of SomeNewClient.
• The services field is underused.  What about flags to say whether a node is a generator, whether it will accept transactions, whether it will broadcast transactions, whether it keeps a peer directory (for the addr message), whether it keeps a full block chain or just headers, whether it should be noted down as a seednode?
• It's possible to request only the header of a block, but not to request only the body.  A headers-only client has to download a load of bytes it already has when it wants to look at the chain in detail.
• Perhaps I've misunderstood, but it seems that getblocks and getheaders both include the protocol version.  Don't we already know the protocol version?
• Again: I haven't looked, but doesn't the alert message allow a DoS?  It's non-trivial to verify a signature, and presumably every alert message has to be checked for validity?  What's to stop an attacker sending broken alert messages all through the network, continuously?  That'll use up plenty of CPU.

Phew. :-)

I'm sure I'm wrong about a good proportion of those, but they're what I thought of while reading the protocol spec.

Wow. Most constructive first post ever. Thanks, realnowhereman.

Or is satoshi airing his regrets? :)

+1

The guy has never posted here and yet he seems to know the code in full details.  Amazing.

I very much which Satoshi could answer those questions.

Satoshi ... please, it would be nice to know your position

Satoshi hasn't visited the forums since December, I wouldn't count on an answer.

I just hope that someday, when Bitcoin has either succeeded or failed, Satoshi can reveal him/her/themselves and take credit for his/her/their work (if he/she/they so choose).

It doesn't eliminate divisibility problems. Obviously the solution is a varint fraction. First, you have the numerator, made up of a variable number of 7-bit sequences (using the high-bit for "more bits to follow"). Then, the same encoding for the denominator. That way you can have exactly 1⁄7 of a bitcoin... ;)

Give it a rest please, Luke-Jr. Decimal numbers work just fine for 99.9999999% of people.

Really, this info is worrying ... Seemingly this forum is the sole way to coordinate and plan efforts.

Is Satoshi backing Bitcoin yet?

Maybe, senior members should drive Bitcoin's future with energy and confidence.

This is already changed, though I don't think I care one way or another. The ports are fixed today so if for some reason you want to find BitCoin nodes you can just check if port 8333 is open.


That would mean you can't start verifying the chain until you have fully downloaded it, complicating the client significantly (you would have to store the chain then reverse it).


That would be nice but not hard to add.


Satoshi didn't give much thought to re-implementations. I am using the subVer field for this in my client. It's unused today except by very old clients, and it allows alert broadcasts to target it if necessary (though I don't have the keys and my code doesn't pay attention to alerts anyway).


It works OK. In practice today all nodes are the same. There's no way you can be just a peer directory, or whatever. Seed nodes are chosen by Gavin/other devs based on which nodes have a good track record of availability, it doesn't make sense to let people elect themselves for that role.


It's only 80 bytes, not a big deal. In practice SPV clients need to download the full blocks after their wallet gains a key anyway. In future a pattern matching protocol will be needed to avoid that overhead.


Yes but you can easily DoS a node with bad transactions already.

That's good.

I'm actually trying to think more generally.  The default port number should be just that: a default.  It really shouldn't make any difference if someone paranoid wants to run it on a different port, in which case, it's nicer if the service doesn't identify itself to anyone who comes knocking.


I'm not sure I agree with that.  What do you mean by "verifying it"?  (I'm only guessing here, I'm pretty new to Bitcoin, but I'm very familiar with git, and the two are remarkably similar).  The chain is verified when you hit the genesis block; but why does the genesis block have to be the hard coded one?  I could hard code any combination of block hash and count in and tell the client that the chain was valid if that block is found.  Further: I can't see any fundamental reason why the entire block chain is needed, surely the very old blocks where all transactions have been spent could be pruned at present -- in which case why can't they be pruned during download, and not downloaded at all?

This idea of "reversing" a chain is pretty crazy as well; the blocks point to their parents, not to their children, so the natural traversal direction is backwards.  If I were designing a structure to hold blocks, I'd have one of the fields in it be "Block *Parent", doesn't that tell you what the traversal direction should be?  Transactions also naturally want to verify backwards since they refer to past transactions not future transactions.


Good.  Here, my thoughts are that this could go someway toward mitigating the 10 minute wait for confirmation.  There is a pre-confirmation level if you know how many peers are holding your transaction for inclusion in a block.  Obviously its not as valid as real confirmations, but it's a long uncomfortable gap between 0 and 1, when it's money on the line.


Well yes; the question was what would you change in the protocol.  I'd change that.  The fact that Satoshi didn't give much thought to reimplementations is the reason its needed.  The existing monoculture of clients is simply not acceptable for a financial service.  It's not acceptable for email servers, DNS servers or bittorrent clients, why would it be okay for storing real money?


"There's no way" at present.  My biggest problem with the bitcoin client is it is so monolithic.  It should be perfectly possible for a client to announce what services it supplies and only supply them.  The fact that there is only one bit in the services field kind of makes my point.

As to the idea that "Gavin/other devs" choose the seed nodes: what kind of peer to peer network is that?  I don't mind hints in the client, but it is really for the peers to decide if they are willing to act as seed nodes, not for developers to pick them.  I'd also envisage it only as a hint to other peers.  If I connect to some random peer and it has announced itself as a seed, I can make special note of that fact and use that node again in the future.  Similarly for answers to getaddr - I would pass seed nodes in preference to non seed nodes.  There is no advantage to pretending to be a seed node, so it's unlikely that people would lie.


I'm sorry, but that's a pretty bad attitude -- if 80 bytes don't need sending, then don't send them.  80 bytes times the 120,000 (approx at time of writing) blocks is 9.6meg that doesn't need downloading.  I'm thinking more about thin clients anyway, and I am particularly interested in maintaining a sparsely populated chain, with the thin client downloading as little as possible at all times.  My idea is that a thin client only needs to verify transactions for its own addresses, in which case a headers-only chain can be traversed backwards only grabbing full blocks along that transaction path.  The full block chain becomes more a local cache than a necessity (and a flag to indicate that that's how the client is behaving in the services field... oops, but there isn't one).


Okay.

I'm not trying to completely slag off bitcoin; there is a lot to admire and it's exciting and new enough a project that it's got me really interested.  That interest has manifested in my working on an independent bitcoin client (as I'm sure are many others).  At the moment though, I'm simply scratching an itch, and this thread was a place to write down those thoughts I've had during my researches.

Yes sorry, to be clear the points I didn't reply to I mostly agree with :-)


Checking that the block contents are valid, ie, that the blocks connect together into a chain that follows the rules and that the transactions within the blocks connect together correctly (are properly signed etc).

You can't do that backwards because then transactions would constantly be referring to dependencies you haven't seen yet.


You're thinking of thin clients that don't do full block chain verifications. Then it could be worked backwards, sure. The protocol was created for full nodes to talk to each other though, and they have to store the entire chain.

It's true that theoretically the chain can be pruned as long as you're willing to take the risk the chain will never fork beyond that point. Transactions that are buried under enough blocks can be deleted as long as the merkle branches of the remaining transactions are preserved.


Seed nodes are only used if IRC bootstrapping fails for some reason. Nodes keep a record of advertised addresses and try to reuse them in future regardless of if they are in the seed list or not. The idea is to allow you to join the network even if the pre-arranged rendezvous point is unreachable and the nodes you've seen before are offline. That's why they are selected based on long term availability and hard coded into the client.


Thin clients cannot verify transactions because they cannot prove there is no double spend. So thin clients need at least some headers, but they only need to download the contents of blocks to find transactions that interest them. A more efficient (but less private) way is to ask peers with full copies to do that selection for them, like polling a mailbox, but the protocol does not support that right now.

Re-downloading 80 bytes is nothing compared to the size of a full block, even today. Avoiding downloading the 80 bytes in the first place, sure, that's useful and can be achieved by bootstrapping from a recent checkpoint if the wallet is empty.

I think realnowhereman's suggestions for improvement are excellent and comprehensive.

The only point I disagree with is that the fundamental unit of currency being "tens of nanocoins" is fine. I find it very surprising that so many people have trouble with converting between a decimal representation of a number of bitcoin and the number of fundamental units.
The mathematics is so elementary.

I'd like to add a suggestion that we revamp the Hash160ToAddress via the Base58Encoding so that leading zeros are not handled in the current peculiar fashion. It's hard to approve of a function that takes a fixed length input and outputs a more human friendly "address" which can vary in length between 27 characters and 34 characters inclusive.
are both valid addresses. It makes the first step of address verification on websites vastly more complex than it needs to be.


There's nothing stopping you downloading the chain in reverse one by one when you get notified of a new block. Your software could also store the chain in reverse and process it with no increased difficulty. There should be support in the network protocol for clients efficiently downloading the chain in reverse. As well as "start height", "version" could return the hash of the last received block to facilitate downloading most recent blocks first.

Not downloading and in fact discarding and forgetting old blocks is of course a central feature of my "balance sheets" proposal.

ByteCoin

PS
That's just a programming problem, you keep track of the unresolved dependencies and check them when you've worked back enough blocks.

That's a reality we already live with. There's little point verifying the block chain before block 105000 due to the "checkpointing" in the current (0.3.20.2) version of bitcoin (see main.cpp).

The overwhelming majority of all hosts that will use this protocol are little endian.

Naively suggesting big endian because it's "network byte order" is simply buying what the Oracle (ex. Sun) and other RISC folks are selling.  It makes zero engineering sense for bitcoin.

However, bitcoin should have been written like proper software, with bytesex conversions, but it was not.


Where are the big endian values?  The network addresses are unavoidable, as is SHA256 words.


Mostly agreed.


This is fixed in 0.3.21.

This also enables accept filtering (TCP_DEFER_ACCEPT in Linux, there is also a BSD equivalent).


It's just a sharing of capabilities.


A little late to change now :)


Uses more CPU power, making it harder to generate a proof-of-work, I imagine.  Also a little late to change now.


Interesting idea.


I wouldn't say "no thought."  satoshi can only do so much all by himself.  There are tons of satoshi ideas still unimplemented...  Just getting the basic, ugly client out there obviously took a ton of work.

Patches welcome!  :)


This has privacy implications.


Agreed.  Too late to change, though.


100% agreed... sigh


Who knows :)  As you note, the time storage is all over the place.


Some software does it like that, so perhaps satoshi simply copied an existing encoding scheme.


Voluminous discussions about rounding on the forums :)

You're not the first to point this out, either :)


Yep, this will be used for future enumeration of capabilities.


Yep (but don't forget merkle details).  Client mode is a key interest for us all, to ensure the bitcoin network remains usable as transaction rates increase.


Welcome to the Department of Redundant Redundancies :)


True.


Nope, not wrong...  bitcoin protocol is... unique.

[/list]

I hardly think I'm naive.  And I hardly see how endianness affects sales of Sun or Oracle equipment.  One or the other endianness could have picked all those years ago (personally I think little endian would have been better -- it's much easier for little processors to do maths with), but what is, is.

C code can and should be written with no assumptions about the endianness of the system.  And I doubt there is very much difference between

x = c[0] << 8 | c[1];

and

x = c[1] << 8 | c[0];

from the processor's point of view.  Therefore "engineering sense" is not really a consideration.

As to your assertion that "overwhelming majority" of clients are little endian; I would suggest it is you who is being naive.  At present it is almost certainly true.  However, have a think about what endianness smartphones are, and how many billion more of them there will be than desktops in five years time -- surely bitcoin should be aiming to be on every single one of them?  The endianness of the platform is irrelevant though, it is simply a standard that network protocols use big endian.  There was no good engineering reason for bitcoin to break with that tradition.


Network address, port number, and SHA256 are unavoidably big endian? There's your answer then.  To be consistent, everything else should have been big endian.


Both good news.


I wasn't saying version was unnecessary, I'm saying that verack was unnecessary.  Therefore the additional complication of coding a handler for verack was unnecessary.  No information is passed in verack that couldn't have been exchanged another way.


Well that's true of a lot of my points.  But the question was "what would you change if you could?", not "what should we change?"

Nothing's ever too late to change of course, if you really want to.  Ready?

return (Block < 500000) ? ripemd(sha256(buffer)) : truncate(sha256(sha256(buffer)))


Which would be a valid point if we were talking about anything other than a checksum.  Most protocols simply sum the bytes to generate a checksum.  CRC would have been a step up.  A single SHA256 would have been overkill.  Double SHA256?  That's the site nuked from orbit.  All those smartphones with their limited batteries are not going to thank anyone for the double SHA256 they have to do on every message they receive.


I'm working on it.

My problem is that it isn't "extra work" that needed doing.  Just different work.  So "Satoshi can only do so much" isn't the argument.


What?  I'm sorry but that's nonsense.

You're saying that transactions that would have been broadcast to me had I been connected to the network are private?  That I can't ask another node "what transactions should I be holding to add to a generated block" because of privacy concerns?  The whole of the blockchain is public knowledge, the only privacy that Bitcoin supplies is the assumed anonymity of addresses.



I'm not surprised.  The problem I expected was that conversion from base 2 "decimals" to base 10 decimals, while simple, is not unambiguous (there are some base-10 fractions that can't be represented exactly in base 2).  I haven't looked though, so I don't know if problems like that are actually problems in real bitcoin.


As I say, I'm very new to bitcoin; but I had thought that the merkle tree is calculated rather than stored (at least publicly).  The merkle tree would be another convenient thing to be able to request; but that's nicely isolated enough that another couple of commands "getmerkle" and "merkle" could easily be added to provide it in lieu of the whole block.  The point is the same though: a thin client should be able to request only those bits of information it doesn't have.


Anyway, thanks for responding.  I understand that these things happen when one is designing something new; hindsight is 20/20.  For all my complaints, I do recognise the core idea of bitcoin is a good one, and it was Satoshi that came up with that, not me.  So what's all my smart-ass criticism really worth, eh?

I feel I was late to the game both for git and bitcoin.  I was working (years ago) on a project that used hashes of hashes to make chains, and never even realised the potential of the technique (I'm certainly not saying I invented it) for distributed systems.  It's funny, I remember my father trying to explain "content addressable memory" to me when I was young, and I couldn't see the point of it.[/list]

I would change it so vital parts of the protocol can be preformed with out a connection to the internet, and only encrypted blocks of cipher text / already signed data would need to touch the internet. People could decrypt and verify signatures on a different machine with no connection to the internet, and the data to try to hash also transferred this way and then coins mined for and stored completely offline prior to a signed/ciphertext transfer of value. This would prevent hackers from being able to attack the network with buffer overflows and similar, root all of the clients and destroy the value of Bitcoin. This could be done either with flaws in the programming of the bitcoin client used, or flaws in the programming of other applications used in a shared environment. I doubt many Bitcoin users are taking security measures capable of defending from intelligence agency / military / super l33t hackers in general and such an attacker could likely take over the network. By removing critical processes from the internet entirely and having only secured/signed/encrypted data online, you can completely remove the risk of hackers 100%. This is the only way to remove such risk 100% as well, but most users are not even securing themselves near as well as they could be while connected to the internet, and the technical expertise required to do this is significantly beyond that of the average computer user.

I should also add that data should be transferred between the internet connected machine and the disconnected machine via CD which is discarded, so an attacker can not use the CD as a compromise vector to communicate data from the disconnected machine to the connected machine and then back to the attacker. Also, at least one back up of the drive of the disconnected drive should be made periodically, incase a compromise attempts to wipe the drive rather than steal the wallet.

A few more:

• If the "version" code prefixed on bitcoin addresses had been anything other than zero, they wouldn't have been a variable length.  At present they can vary between 27 and 34 characters.
• The "sequence" number is stored in the TxIn record.  If I've understood, the sequence number allows us to replace one transaction with a later version, why then is the sequence number in the TxIn field?  Surely if we change the inputs we'll change the outputs?  The sequence number should be part of the transaction record, not the txin record.
• OP_CHECKSIG:  Woah!  It really doesn't seem right to me that a script operator is so complex.  It just feels wrong that the scripts need to be filtered, truncated, and reencoded into a copy of the transaction; and this is done for every TXIN.. and comes up with a different result.  I understand the problem, every component of the transaction needs signing as a whole by all the owners of the source TXOUTS, but OP_CHECKSIG reads like some sort of frankenstein monster.  My first instinct is that the signatures shouldn't have been part of the script, they should have been indexed fields in the transaction header, referenced with an OP_FETCHSIG(1) or similar in the script.  That way the script would be static and the signature could remain outside the block that it is signing.  I suppose to be more generic it would have been better to have a simply parameter block as part of the transaction header, and the operator would be OP_FETCHPARAM; but the idea is the same.  It's not like this is even out of character.  OP_CHECKSIG already requires out-of-script data.

I would like to see address fields support host names and not just ip addresses.

IP transactions are disabled already, I believe. In any case they really shouldn't be used.

I don't mean for transactions, I mean for Peering.

Ah, then yes that would make sense, though I don't think that's protocol, just implementation.

Its protocol since the addr record only supports ips but it would be easy enough to add another type. Also, it would solve the problem of a nat client not knowing its own ip. You could just enter you dyndns address (and port because I don't like the fixed port.)

I agree with this one. Was it really necessary to implement a whole scripting language for a single-purpose application like bitcoin? I think it would have been better to go without the scripting part, and just keep things as simple as possible. Right now the code is so complex, that even seasoned C++ coders do not understand it, and everybody has to defer to the judgement of a few specialists. I think the simpler the code-base and the more people understand it, the greater the confidence that the implementation is sound.

Right now for me the code is just one big obscure blob. But then I'm not a good programmer, so maybe it's just me.

It's existence is a nice solution to dealing with future extensibility of bitcoin; but like so many things in bitcoin, the idea is excellent, but the implementation is... complex.

Apart from a few obvious exceptions, the scripting language is actually fairly simple; so much so that I wonder what possible future it is defending against.  The main problem I see is that the interface between script and transaction is opaque.  I would rather have seen the likes of OP_CHECKSIG implemented as simpler component part in the scripting language, with other operators that fetch transaction related information as needed.

OP_CHECKSIG and friends stand out as being of a ridiculously high level in a fairly simple language.  It might as well have been one operation per transaction: OP_DO_ALL_THE_STUFF_THAT_HAS_TO_BE_DONE.

All that being said, I kind of feel myself edging towards your point of view... was the scripting language really necessary?  Especially in the form it is.

It's necessary for distributed contracts. There's no GUI for that in BitCoin today, but over the next few years it's an area that will be explored more.

But was it necessary to be in the form its in now?

I doubt it.

I don't really know what a distributed contract is though, so I don't suppose I'm entitled to say.

There are some really interesting transactions you can make using scripting.

A good explanation by Satoshi: http://bitcointalk.org/index.php?topic=195.msg1611#msg1611

edit... Another, more recent thread: http://bitcointalk.org/index.php?topic=6900.0

In both of those threads, it's pointed out that the "clever" idea doesn't need any special scripting, and the ideas are better done in another way (which didn't involve scripts).  That still pulls me towards the conclusion that scripting in its current form is simultaneously overkill and underkill.  It can't do some things that may be wanted, and it does plenty that isn't.

As I said, I can see the potential in the idea of scripts, I just think the implementation is wrong.  Apart from a load of operations that are unlikely to ever be used (OP_2ROT?) in transactions as they currently are; there are a small number of commonly used operations, and one key one (OP_CHECKSIG) which embeds so much intelligence in a single operation that it might as well have been a flag early on that said "handle this transaction in the usual way".  And the nonsense that is "template matching" of scriptPubKey for spenders to detect what the sender wanted them to put in scriptSignature demonstrates my point programmaticly.

Like so much of Bitcoin, I think the ideas are revolutionary, but the implementation leaves a lot to be desired.

Rather than be one of these drive-by-moaners though, I've become interested enough that I'm working on my own independent Bitcoin code.  As much as Satoshi and Gavin think it will be "a menace", I disagree, and think a cleaner design will allow for easier derivatives.  I know, I know, that's what they all say :-)  No programmer ever wants to write a patch do they?  They'd rather start again on their own.

My ideas:

• Separation of GUI, protocol and wallet
• Librification
• Smartphone implementations
• Desktop "light" client
• Sparse block chain supprt
• "Walletless" node
• Pluggable miners
• More flexible wallets
• More portable wallets
• Strong and easy backward compatible protocol version support

There seems to be a need for many of these, and many of them seem to have been addressed by creating a custom client.

Whether I'll succeed or not remains to be seen; and I certainly don't want to implement all of the above myself.  However, I would like to create a solid enough base that is modular, clear and easily extensible enough for others to get involved and implement what they want more easily than they currently can.  I've browsed the current client code and (sorry if this upsets anyone) it is an unholy mess.  I don't ascribe blame or condemnation for that -- the first version of every new system is always messy, as they tend to grow organically.  As the mythical man month recommends though, "plan to throw one away, you will anyway".

I'm not ready to share yet, but what I've got is going well, and isn't an unholy mess.  Most of my comments here are notes that I've made during coding as I stumble across oddities.  Perhaps I'll hit a wall of complexity that I cannot clear and will give up; so far I don't foresee anything that is going to insurmountable.

realnowhereman,

it's great that you are working on an alternative implementation. I was thinkining of doing something like that too, but I guess I am more of a drive-by-moaner than a programmer :) And I don't think it is a menace to have an alternative implementation - I would actually rather put my money in a clean clear and concise implementation of Satoshi's idea, rather than the current huge blob of C++ code.

I was thinking of using python to program the application, and use twisted for the networking, and protocol buffers for the messaging over the network.

Can you share with us more details of what language you are going to use for your implementation, and what external dependencies are you going to introduce? What will you use for the GUI? Are you going to keep the scripting, or will you do away with it?
 

I'm conscious that I'm going off topic now, so forgive me forum admins.


I wouldn't object to any language, but I feel that C/C++ give the most flexibility for a library to be integrated in other languages.  I like object oriented, so C++ it is.


As few as possible.  Personally I don't like libraries that pull in a mountain more of dependencies, so where it is reasonable to do so I've coded my own library modules.

For the library/server/command line parts it's looking distinctly like the dependencies will be OpenSSL for the hashing and ECDSA; and maybe boost for some reference counted pointers, but if the pointer dependencies stay as simple as they are now, then I'll drop that and use explicit memory management to save the dependency.


I use Qt when I have to dirty my hands with graphical interfaces.  But my main effort is in the creation of a useful bitcoin library, so there will be strong separation between the GUI and the protocol.


I'm not planning to fork the protocol and will be aiming for 100% compatibility with the official client.  I have no desire to take over from the official client -- in fact that would be counter to one of my fundamental reasons for starting this: there shouldn't be a monoculture of bitcoin clients.  There is danger in that, so I'd prefer to see multiple competing clients.

Be aware of the dangers of reimplementations too. If your client diverges from Satoshis code even in a tiny detail you risk forking the chain (if people use your code for mining).

Any reimplementation usable for mining that did not come with a very, very large test suite would be a worrying thing indeed.

Mining is pretty low on my priority list as it happens, so it's won't be a problem for quite some time.

Perhaps I haven't understood how mining works, but I thought it was simply a matter of adjusting a nonce until the hash of the block header has a particular number of zeros at the front of it?

As to forking the chain; since the official client will not accept blocks that don't meet its criteria, it's unlikely that a broken implementation could gain traction, since the blocks it mined would never make it into the majority voted true chain.  Unless of course 50% of miners instantly switch to the new miner code.

If it really is as easy as you suggest to fork the chain, then Bitcoin has a serious security hole in it.

Yes, the risk is that you end up with lots of people using a slightly different implementation. It doesn't have to be 50%, just a significant fraction. The worst case scenario is that one day somebody uses an exotic or badly tested feature and the chain permanently splits because the two implementations disagree on what's valid. That could potentially be an absolute disaster for the system, perhaps one of the few things beyond being made illegal that could instantly kill BitCoin. Your view on the economy would depend on what software you ran.

realnowhereman: Great work, you nailed most of my concerns with the protocol.
I can see how it was quickly prototyped but we cannot continue to use this monster - it is not consistent and/or logical at all!

I read a lots of "It's too late to change that" and whatnot - no it's not! Thats why you support protocol versions. If the version message itself needs to change, then realnowhereman had the answer to that with block based switch.

One thing that has not been mentioned - please pretty please just drop the checksum from the packet - it has no function. Early with IPv4 many people though that this would be a problem so everyone implemented checksums on all layers of OSI. Today this is assumed to be handled by the physical / data layer - IPv6 doesn't have any checksums for instance. Even if you do keep it, why on earth do sha256? It has been said and I will say it again - CRC or something that does not require heaps of computing power would be soo much better.

These issues annoy me to the brink that I'm thinking about forking the client and creating a new protocol and talk the new protocol with those that support it. It's the slow way to revolution but it is very possible.

Any incompatible change has huge consequences -- users with older clients will not be able to spend their money.

We will continue to avoid such changes until circumstances demand it.

That is only a problem if you do not dualstack.

Any change can't be done easily and each design decision (protocol or implementation) can be source of conflict.
I had so much change to do that it was impossible to merge them with the current implementation, the main source of problem
was to make a client only version (that was the early goal of the bitdollar client)
Instead i started working on a different protocol (i have posted the main differences with the bitcoin protocol http://www.bitdollar.org/forum/viewtopic.php?f=8&t=14 )

There doesn't seem to be any problem requiring that users have the latest version at present....

I'm thinking particularly of the minimum transaction fee being hard coded at transactions smaller than 0.01.  Then being hard coded at 0.0005 when that was too large.  Next week it'll be hard-coded to something smaller again, I have no doubt.

I'm a bit uncomfortable with all the hard-coded magic numbers in the Bitcoin client, for exactly your reasoning: when they change they need a new client not just a config file update.

You are confused.  There is no such requirement.

Plenty of clients remain on 0.3.19, and you can still find clients from earlier versions.

The earliest bitcoin client may still log onto the network and spend their bitcoins, at this point in time.

I'm not confused, the requirement is implicitly made by having hard coded values.  I obviously didn't make that clear.

Tell me, now that bitcoins are approaching $20 how these 0.3.19 clients send me 19 cents without a 100% fee?

They set their transaction fee to 0.00 BTC and then send you a transaction. It may take a while to confirm, but you'll receive it.

Tell me, other than with Bitcoin, how can you send $0.19 to someone electronically without paying a 100% fee?

Yes, but the point was to have a small fee that can be sent as a reward for the verification.
It's not very nice if you're only left with the choice between sending a huge fee (BTC 0.01 when USD 1 = 1 UBC) or none.
That's not the point. One of the intentions of the BTC is to make micropayments easy and cheap. USD 0.19 (or more) isn't cheap :)

On the other hand, if you'd parameterise (sorry for my UK English) the fee, who is going to decide about a new fee? Some centralised agency (Satoshi and/or Gavin)?
Or the program itself, based on ... yeah, what exactly? Exchange value of the BTC/USD?
Or maybe some kind of closed feedback loop with a fixed (hehe, here we go again: why not also parameterised?) difficulty setpoint?  :-\

I think you're missing the point; and these sorts of blind defenses of Bitcoin don't make it easy to discuss the subject of reasonable improvements rationally.

I'm not attacking Bitcoin; I love Bitcoin, that doesn't make everything perfect though.  I'm saying that, despite protestations to the contrary, a later version of Bitcoin is needed for the whole network; and that this idea that older versions are fully compatible with newer versions is wrong.  If I want to perform the send I gave as an example, I can just get 0.3.22.  So it's not Bitcoin versus $NOT_BITCOIN; it is 0.3.21 versus 0.3.22.

My wider point is that problems like this will continue to turn up while the client has so many hard-coded parameters.  If you want a different example, let's talk about the upcoming removal of the maximum block size limit.  When some clients are making 5MB blocks and those are rejected by older clients, is this pretense that backward compatibility is preserved still going to be maintained?

Here are some of the other hard-coded values I've come across.  I've given them symbolic names, but in the official client they are just dumped at whatever point is appropriate in the code.


And I'm nowhere near done.  So: putting these things as literal instead of constants is poor form; but I suspect that quite a few of them should be in a configuration file.  When it turns out that one needs changing, a whole new version is going to be needed.

The problem is exemplified by the current need to change the minimum transaction fee that I keep banging on about.  If this were in a config file then nobody would have to wait for the next version to be able to keep up with the market.  It surely can't be a good thing that the client needs changing depending on what the current Bitcoin price is?

Finally: to preempt the idea that hard-coding these protects the network in any way: a motivated attacker will have no trouble downloading a compiling their own version with whatever parameters they want set.  So all the hard-coding does is make problems for honest users.

No, it's not.

Even satoshi's original paper and presentation mention that bitcoin is not the greatest for micropayments.

If you're wanting the actual number of blocks (including the block with the coinbase) this is the correct value.

Client also hard limits it to being ignored for 120. Very confusing. But for very good reasons. Prevents some rare chain split / double spend scenarios. Changing that one is actually not in your interest.

From official client; main.h:

This thread deserves bumping for great justice.
Especially now, that Bitcoin network is growing exponentially.



/me likes this.



+1

Edit:
I need to be very humble because you people have put so much effort, ingenuity and skills into this program, and I'm just a newcomer/user of the system, but I really have fun taking part in this kind of conceptual discussions.
So I wanted to say that I think that probably you are right and Satoshi is right.
And I think we can make it better, no? (Oops, excluding myself from the 'we', because I'm a very poor programmer.)  ::)

After checking his whitepaper I found this:
Mentioning this as a disadvantage for 'old' currency in a paper in which he proposes a new one, means for me that he intends the bitcoin to perform better in that respect. Sorry if I missed later posts on this forum in which he states otherwise, but that's how I read it.

And a micro-payment requires a micro-fee, so I think the only fixed parameter for the minimum payout of a fee, would be the minimal denomination of the bitcoin, i.e. something like 10^-8 BTC(?).

Yes. But that doesn't include the block that has the coinbase. So it's actually 101 before it matures, and 120 before it shows in the gui or bitcoind as matured and it wont let you spend it until then.

Interesting information. But all I was saying was that 100 was the number written in the source, so saying it should be 101 is not true. If that constant were changed to 101 then it would need 102 blocks to mature.

Well, I'd want some very fundamental changes, probably only realizable in a fork. Please have a look:

http://www.newbitcoin.org/documents/newbitcoin.pdf (http://www.newbitcoin.org/documents/newbitcoin.pdf)

I like some of your suggestions, but I am not sure if the new transaction fee structure you propose will work. Also I am not really sure I understand how the new block-chain will work.

I don't think you are correct in claiming that

I don't see any significant shortcomings in bitcoin that will lead to its collapse. I think it suffers from the fact that the protocol is still not well defined and the implementation is overly complicated, but there are no major flaws that will lead to its collapse. The main idea behind it is very sound.

For any successor to bitcoin to actually have a following, I think it needs to provide some minimal backward compatibility with the current client. Even if a completely different protocol or a different blockchain is used, at the very least the current wealth distribution must be preserved, or there must be a smooth path to move current bitcoins into the new bitcoins. Lacking that, the new currency will probably have to start trading from zero and compete with the old implementation of bitcoin.

Thanks, I'm very easily flattered! It could very well be that I didn't explain it clearly enough, but could you please be more specific about what part you don't understand?


It'd be quite easy to defend that the system MAY collapse, but I won't be so bland. Some quotes, maybe that'll initiate some proper discussion:
- Higher transaction fees lead to slower transactions
- Enforced limits are not optimal
- Not truly decentralized
- Unmanageable storage requirements
- Technical difficulty
- Current distribution is unfair

We could discuss about where the 'main idea' (which I too find very sound) stops, but until these issues are addressed I think bitcoins will not get much further than a nice new subject of unfounded speculation for the not-so-technically-literate.


You are absolutely right, I'd be happy to be convinced otherwise (I happen to have some bitcoins myself), but at the moment I think starting from zero is the only way. I remember reading in this forum (or in some documentation) about some smooth transition policy.

Most of these issues are what make me start a new protocol.

Thanks for writing an interesting paper Steve. I need to think about some of its proposals more, but here are a few early comments:

• Bitcoin already defines the winning chain to be the chain of highest total difficulty, not the longest.
• I think you are over-concerned about storage costs (I think Satoshi was too). Storage is incredibly cheap and the price has been falling for a long time. When I look at the costs involved with running a high speed Bitcoin node, the bottlenecks are things like CPU time and disk seeks, not number of bytes storable.
• Removing script and simplifying the transaction format is superficially attractive, but would eliminate a lot of useful features that are helpful for a non-trust based financial system. The script language isn't really that hard to implement or review. Bitcoin is still millions of times simpler than, say, a web browser. My gut feeling is that Satoshi made the right flexibility:complexity tradeoff here.
• The idea of allowing peers to peel off fees for themselves before relaying is very interesting. I think this is the strongest part of your proposal. I also suspect it's possible to implement on top of Bitcoin using the features that are already there, but I haven't thought about it enough to prove it.
• Early adopter wealth might seem "unfair" but I don't think that's enough to convince people to switch to another currency. People who used Bitcoins when they were worthless deserve a reward for their efforts. Nobody is going to end up owning big chunks of the planet because it's so very unlikely Bitcoin will fully replace existing currencies within our lifetimes, perhaps never. Much more likely is that it finds a niche in the electronic payments space, grows to fill that niche then stabilises.
• I didn't really understand your proposal about chain difficulty. Blocks have a cost associated with them, even though it's small, so allowing people to flood trivial blocks that then have to be stored and verified doesn't seem to help. It's just another DoS avenue.

Don't get me wrong. I think it's great that people are exploring evolutions of the Bitcoin design.

Thanks Mike. I was just starting to think every developer was too busy following mtgox...

What i'm missing is clear, unambiguous specs. But maybe you have a link?

I agree that you probably know best what the bottlenecks are. But I don't agree storage isn't a problem. Say one transaction is 0.2 kb, suppose we want Bitcoin to be used by millions of users, say 100 million. Then storing that one transaction costs at least 20 Gb. Which is not good enough for micropayments I'd say.

My gut says it's too complicated, but you might be right. Haven't seen a killer application yet, but must admit I could do some more studying on this one.

Think it basically comes down to including a possibility to send private keys along with transactions.

Then for your sake I hope your right. As long as you understand that a completely 'fair' system, as described, is very feasible too.

I wanted to get rid of arbitrary limits (maximum block size, difficulty) and 'time'. There will be more chain-forks, maybe chaos and anarchism (up to a point: the specification). But don't believe flooding will get you far, I expect an optimal equilibrium difficulty.

Sorry i hadn't read all your document in depth, it's done.

I would join you unfortunately i'm already working on the implementation of another protocol,
here the differences between the issues you point in the document, and how i handle that:

Higher transaction fees lead to slower transactions:
That's right, but it is a problem only if a large majority of nodes do that and if some does:
Since transactions are spread all over the network, only the transactions' hash are transfered in blocks.
Nodes are supposed to already have the transaction in inventory, if they dont have, they wont validate
the block until they have received all.
Low block spread is bad for all nodes, but the block of someone that kept some transactions will be transfered
over the network slowly compared to others and could lose race.

(there is also a system to extend the transaction fee of an emited transaction, which could be used to
"award" nodes having it, if client see that transaction has been correctly spreaded)

Your way to solve that problem is good but each transaction would need to access to more accounts which have a cost in disk IO
also i use a one time signature scheme (using hash chains) which can't do that, an emited transaction can't be modified.

Enforced limits are not optimal and Not truly decentralized:
i haven't put limits, limits means doubts in the protocol capacity.

Unmanageable storage requirements:
Huge storage needs also means huge bandwidth needs.
By reducing this, we increase the network number of nodes, strenght
and reduce the transactions fees.

The way i manage accounts and transactions is different, but final result is similar (transactions are only stored 3month, all is kept in a list containing all accounts).
I differenciate account and addresses (address = public key, used to create an account).
Very old accounts without activity also have a small fee each year (these will be mostly account of users that have lost their private key)

I also think storage is a problem since it is the easiest way for someone to attack the network.
For example, without being paranoid, someone could use 1million $ to create transactions that will
take the most possible definitive storage in the chain, that could make serious problem to nodes.
And it could be far more, mass storage capacity is part of the network security.

(Assuming the network is a real p2p network and that nodes have average user disk storage, if there was only
few hundred nodes maybe they could handle, but the currency would not be "p2p" anymore)

No unnecessary difficulty:
I have keep the way bitcoin manage the block race, but with some optimizations and more challenge
between nodes that spread the block.

Proof-of-work as ’currency’:
the way b$ is done, i could not handle a non fixed number of units.
instead i have made the award proportional to the target a of block

Wow, I didn't expect to see my old thread when I browsed this forum today. To be honest, I took a pretty long break from bitcoin, and only recently began thinking about it and reading the code again. In fact, Sunday was the first day I even ran the client! That being said, allow me to do my best to summarize most ideas shared in this thread.

Keep in mind that this thread was not just about ideas that could potentially make it to a future bitcoin release, but ideas that could be used in an entirely new protocol. Everyone will benefit from competition in the cryptocurrency market.

1. Data Serialization: Vladimir (http://forum.bitcoin.org/index.php?topic=4278.msg62596#msg62596) suggests using BERT/YAML for binary and text serialization, and comboy (http://forum.bitcoin.org/index.php?topic=4278.msg63724#msg63724) suggests JSON for the latter.

(I'm putting this first because even though it's on page 2, I believe it's the biggest flaw in the current Bitcoin protocol, and a reason for many other problems listed here. Bitcoin is largely a binary-encoded protocol, so the composition of packets are set in stone. A more fungible serialization format like the aforementioned ones (or better yet, protocol buffers (http://en.wikipedia.org/wiki/Protocol_Buffers)!) would allow the addition of new fields that older clients could safely ignore. All this talk about client versions, encryption algorithms, timestamp lengths, and more, would be much less of a worry, so I'll include them as 1a through 1d.)

1a. Client Handshakes: Cdecker (http://forum.bitcoin.org/index.php?topic=4278.msg62278#msg62278) and realnowhereman (http://forum.bitcoin.org/index.php?topic=4278.msg95980#msg95980) suggest making it possible to specify the client verson separately from the protocol version. grondilu (http://forum.bitcoin.org/index.php?topic=4278.msg64029#msg64029) suggests allowing a client to specify what algorithm they use for their key. realnowhereman also suggests flags to say whether a node is a generator, whether it accepts transactions, etc.

1b. Integer Lengths: realnowhereman (http://forum.bitcoin.org/index.php?topic=4278.msg95980#msg95980) suggests cleaning up the arbitrary use of 64-bit and 32-bit integer lengths; for example, having timestamps be 64-bit consistently.

1c. Hostnames: just_someguy (http://forum.bitcoin.org/index.php?topic=4278.msg103737#msg103737) suggests supporting host names alongside IP addresses.

1d. Transaction Scripts: realnowhereman (http://forum.bitcoin.org/index.php?topic=4278.msg103557#msg103557) and alkor (http://forum.bitcoin.org/index.php?topic=4278.msg104751#msg104751) suggest getting rid of the complexity of the custom scripting language, and implementing a simpler system.

2. Byte Order: error (http://forum.bitcoin.org/index.php?topic=4278.msg62130#msg62130), Cdecker (http://forum.bitcoin.org/index.php?topic=4278.msg62278#msg62278), and realnowhereman (http://forum.bitcoin.org/index.php?topic=4278.msg95980#msg95980) suggest standardizing on big endian, to be consistent with the native byte order for network addresses and hashes, and to match some mobile platforms. Others, including me (http://forum.bitcoin.org/index.php?topic=4278.msg62138#msg62138), jgarzik (http://forum.bitcoin.org/index.php?topic=4278.msg62161#msg62161), and xf2_org (http://forum.bitcoin.org/index.php?topic=4278.msg96617#msg96617) suggest little endian to match the most common computing architectures. Not entirely sure what side I come down on now.

3. Coin Divisibility: genjix (http://forum.bitcoin.org/index.php?topic=4278.msg62185#msg62185) suggests using INT128 to allow Bitcoins to be more divisible. realnowhereman (http://forum.bitcoin.org/index.php?topic=4278.msg95980#msg95980) suggests a fixed-point base 2 integer. Luke-Jr (http://forum.bitcoin.org/index.php?topic=4278.msg96175#msg96175) suggests a varint fraction that specifies the numerator and denominator separately. ribuck (http://forum.bitcoin.org/index.php?topic=4278.msg96186#msg96186) suggests he give it a rest please =)

4. Block Size Limit: caveden (http://forum.bitcoin.org/index.php?topic=4278.msg62216#msg62216) suggests having the protocol automatically adjust the block size limit according to the transaction rate.

5. Block Discovery Interval: dirtyfilthy (http://forum.bitcoin.org/index.php?topic=4278.msg62583#msg62583) and comboy (http://forum.bitcoin.org/index.php?topic=4278.msg63724#msg63724) suggest lowering the expected block discovery interval, to take into account the fact that networks speed up over time.

6. Hashing Algorithm: Vladimir (http://forum.bitcoin.org/index.php?topic=4278.msg62596#msg62596) and comboy (http://forum.bitcoin.org/index.php?topic=4278.msg63724#msg63724) suggest using multiple hashing algorithms for block generation. realnowhereman (http://forum.bitcoin.org/index.php?topic=4278.msg96769#msg96769) suggests the opposite: use a trivial CRC to make it quick for mobile devices to verify blocks. Difficulty could still be increased by increasing the number of 0-bits required in the beginning of the hash, or even increasing the number of iterations required as I suggested (http://forum.bitcoin.org/index.php?topic=4278.msg63704#msg63704).

7. Block Downloads: realnowhereman (http://forum.bitcoin.org/index.php?topic=4278.msg95980#msg95980) and ByteCoin (http://forum.bitcoin.org/index.php?topic=4278.msg96383#msg96383) suggests making it possible to download recent blocks first.

8. Misc: To save space… =) realnowhereman (http://forum.bitcoin.org/index.php?topic=4278.msg95980#msg95980) suggests: (1) not specifying one's own IP, since such information could be inaccurate if behind a NAT; (2) getting rid of the unnecessary verack and use of RIPEMD; (3) a client that works in a multi-user environment; and (4) the ability to query a node for transactions it has queued.

Thanks for all the great ideas, everyone. I didn't have time to get a close look at Stevie1024's paper, so I'll do that later. I haven't decided whether I personally want to work on a Bitcoin client or an entirely new protocol, but I really hope either way we end up with a better, more robust cryptocurrency system in the future.

oskar, thanks for posting your summary just as I stumbled upon the thread!  Good thing I started reading it at the end.  More suggestions:

Alternative chains

I would like to make it easy for distinct proof-of-work chains to share hashing power along the lines Mike proposed here: https://en.bitcoin.it/wiki/Alternative_Chains

I haven't completely thought through the details, but I think the "headers" message (or a successor) should support an "indirect" header in addition to the standard kind.  The indirect header would come with a standard header from another ("main") chain.  This "main" header would hash to a value below the target in the alternative chain, and the indirect header message would include:

• the main header's coinbase transaction, whose script would include a Merkle root of headers from different chains
• the Merkle branch anchoring the coinbase transaction to the main header
• another Merkle branch anchoring the alternative chain's header to the root contained in the coinbase script

This would avoid the need to fragment hashing power as is happening between Bitcoin and Namecoin.  This would encourage a proliferation of experimental currencies and other applications, which would be a Good Thing.

The original BTC block chain would not necessarily have to accept indirect headers, though it would be nice to have it share logic with chains that do.

Pluggable policies

Subtle differences in the block acceptance rules in use among miners threaten chain unity.  When (not if) a block appears and some, but not all miners accept it, confusion will ensue.  I would suggest a protocol field encoding the block acceptance rules.  It could be, for example, a hash of a Forth or C++ program.  Miners could obtain and compile (or interpret) the program when they notice they are hashing on the losing side of a chain split.  Obviously, they'd want to restrict this feature to avoid malware, so C++ is probably not the best choice.

I call it "pluggable policies" because I imagine changing the client to support loading and using multiple block acceptance policies for different chains.  This is not strictly a protocol change.

I've made the suggestions of protocol buffers before (http://forum.bitcoin.org/index.php?topic=632.msg7306), I still consider a non hand crafted binary serialisation protocol almost vital for bitcoins to really take off.

Satoshi wasn't a fan of the idea, I wonder what the new development team thinks?

Well, if we are talking about a do-over, I'm pretty sure satoshi said something like "if I had to start from scratch, I'd use Google Protocol Buffers" and I agree with that.

On a slightly different subject, I would use RSASSA-PSS instead of ECDSA for keypairs.

Is there a reason a protocol buffers based protocol cannot be implemented as an additional RPC mechanism and the old mechanism phased out?

Yes, plain ole backwards compatibility.  We had never broken compat with older clients, and I'm not rushing to start now.

And if you aren't breaking backwards compat, that implies forever supporting two P2P protocols instead of just one.

Changes to message formatting and byte-ordering do have a lasting effect on the hashes that are used throughout the network. So everything that ends up being hashed is set in stone, unless we want to run dualstack for a really long transition period (until the last input from the old protocol has been spent, which probably will never happen due to lost wallets...).
Having to adhere to the existing protocol for large parts makes other changes pretty useless, as sad as it sound...

What I would like to do is add some structure to the network in order to reduce message complexity (think DHT for transaction inputs instead of everybody tracking everything), detect network partitions and a more hierarchical network topology (miners in the center, lightweight clients at the edge).

Excellent summary.  With one minor nit:


I certainly don't want to weaken the algorithm for the block hash (although I have my doubts that it needed to be double SHA256, but that the fact that blocks need to be mined already means that this is irrelevant other than making it slightly more computationally intensive to verify a block).

What I was suggesting was a considerably simpler algorithm for the payload checksum in network messages.  At present it's the first four bytes of the double SHA256 of the message payload.  This is complete overkill.  I'm not entirely convinced that any checksum is needed -- checksumming should be (and is) handled at a lower level in the network stack (ethernet, TCP, PPP, etc, all include checksums in the packets already, so why checksum again?).

Think about this: when was the last time you saw a single byte received wrong in a web page you were looking at?  Well HTTP includes no checksums, and relies on the TCP checksum entirely.  Seems to work okay.

Checksumming and the message magic bytes seem redundant, that's what Mike and I where thinking. Probably Satoshi was hunting down a bug in his code and introduced them. I love the magic bytes however since it enabled me to track down a bug in my protocol implementation (non-blocking IO and not yet completely filled read buffers :D). A simple CRC should be enough for checksumming, if we want to have checksum at all. On the other side being a non-trusted network, and since any client may bomb you with anything, the checksum is not needed at all :D

The magic bytes and payload length are important. They allow you to stay in sync when you are sent unsupported messages, and recover if you are sent junk (i'm sure i read that the official client does send junk sometimes).

There's an incentive to do so, and a good protocol should take it into account I think.

How to deal with 'old accounts' (or the growing number of accounts) is definitly one thing I don't have a proper solution for.


I am very curious about what you're developing and the solutions you have. Since most of the above seems to involve your protocol, I will respond on your b$ site or in a pm. One general remark: I don't think trying to develop a new protocol on your own is a good idea. I hope there will be an initiative quickly, in which more developers will assemble and start writing together.

Hope you do and you will agree. Also hope you will decide it's time for a new, sound protocol, for me the decision is obvious. And I hope there's more developers thinking the same way, cause I'm quite sure I can't get nowhere alone.

Personally, I'm more in favour of evolution than revolution.

I'd first like to get a more programmer-friendly client out there (which is what I'm working on, slowly).  Hopefully that will make it easier to run experimental chains such as Stevie1024 would like; easier to integrate with website; easier to write alternative GUIs; and easier to store keys in whatever format is suitable for your business (I'm envisaging keys stored in SQL databases for large web stores).

However, while I see plenty of faults in the bitcoin implementation (as I listed above); the technology itself is sound -- let's not kid ourselves that the endianness of the bytes, or the use of binary instead of YAML affect the operation of bitcoin in any way at all; they are mechanism rather than policy.

The current bitcoin protocol is what it is.  Network effects mean it's not worth making a huge effort just to add one extra byte of precision to a timestamp and making old clients incompatible.

Wow, Mike, are you still sure about this then?

The scripting language is fairly trivial, it's not what makes Bitcoin complicated. The complexity comes from things like ensuring you correctly implement all the verification steps, time handling, dealing with re-orgs, detecting dead transactions, being able to catch up given only the chain and some keys etc.

OP_DROP and friends is a small amount of code that's easy to unit test. It's also not required to run scripts at all for lightweight/SPV clients.

Response to OP:

Protocol needs a financial system. Would issue a large fraction of coins as contingent claims on future difficulty states rather than purely in an instantly maturing form. This system would make it easier for entities to realize profits from actions that influence future generation difficulty (e.g. adoption by large businesses). It would also reduce the costs of hedging possibly significant (but not catastrophic) changes in the price level. In the current system, rewards go mainly to early adopters (essentially rent-seekers). Don't care about the distributional issues, but as an incentive system the current coin generation protocol is wasteful.

1. transaction fee with a difficulty such that block growth is set at maximum 1 mb per day.
2. auto clean function on January 1st, of each year to compress blocks such that the only info kept is the owner of each BTC.  block0002011.dat, now is retired for block0002012.dat.  Stop transactions December 31st to completely verify new block.
3. my as well add namecoin to it with possible 1mb/webspace.  Allow you to keep webspace forever, if you pay a deflation fee to the network.
4. put BTC growth on steroids such that most BTC will be out in 4 years.  Halve every year instead of 4.
5. change to a cpu as oppose to a gpu based problem.

This is a great thread, I have just started looking into the details of Bitcoin but it appears that the protocol is not designed with crypto agility in mind.

Given the nature of cryptography, e.g. that all crypto systems eventually become obsolete I would say that a crypto curency would need to be crypto agile.

In otherwords, one should be able to move from one algorithm to another without requiring revising the whole protocol; there are many ways to approach this problem in the case of ECC one might at a minumum plan from migrating from secp256k1 to other curves like secp256r1, secp384r1 and secp521r1.

Also since there has been significant advances in cryptographic attacks against hashing algorithms (though the SHA2s are likley safe for a very long time) having a conceptual model of how one would support SHA-3 at some point would be good as well.

Ryan

The hard limit on supply of bitcoins is the biggest problem IMHO.  The deflationary spiral threat is well documented so I won't repeat it except to point out that it gives bitcoin far more characteristics of an asset rather than a currency and provides a disincentive for circulation.

Monetary policy should be internally regulated and dynamic.  I'd propose something along the lines of a feedback loop where the rate of coin generation is proportional to the incentive to hoard coins.  Thus if you have excessive deflation more coins are produced, if you have inflation less coins are produced.  The hard limit would disappear and the real world value of the coins stays relatively stable.  The difficulty is that you can't tie it to any externality like exchange rates because then you need gateways to those externalities which creates points of potential manipulation. 

My first thought was to tie it to the inverse of the rate of circulation which is easy for any node to calculate.  The more people are hoarding, the more new currency is added to the market thus devaluing the hoarded currency or at least finding an equilibrium and removing the hoarding incentive.  But that would need to balanced with a disincentive to spam transactions or the rate of circulation could be manipulated (and probably choke the network if the effort was big enough).  My next thought is to tie it to the number of unique coins that have been transacted in the block (or the average of the last n blocks) compared to the total generated.  Each coin can only be counted once no mater how many times it is transacted thus preventing spamming with circular transactions to manipulate the rate.  Very large holders could still have an effect by ensuring their coins were moved at least once every block so perhaps it could actually be a one way test:

if circulation > threshold --> create normal number of new bitcoins
if circulation < threshold --> create normal number of new bitcoins + (threshold - circulation) * factor

Obviously not a fully fleshed out idea but hopefully greater minds can add to it...

further thought, miners must have an incentive to process transactions (transaction fees) to counterbalance the incentive to hold them back to reduce apparent circulation and increase rate of coin production.