Bitcoin Forum
March 29, 2024, 03:32:22 PM *
News: Latest Bitcoin Core release: 26.0 [Torrent]
 
   Home   Help Search Login Register More  
Pages: 1 2 [All]
  Print  
Author Topic: Bitcoin protocol standarization  (Read 3819 times)
lontivero (OP)
Full Member
***
Offline Offline

Activity: 164
Merit: 126

Amazing times are coming


View Profile
November 28, 2014, 05:04:22 AM
 #1

Hi,

Everytime I tell some people about bitcoin they ask me "who controls it?". I tell them nobody does it because it is a protocol just like http or smtp. However sometimes I think that that is not true. Of course nobody controls the currency but the bitcoin core team is who takes the decisions about the "reference" implementation, what means it is the team that update the protocol even when there are other full-node implementations out there.

In fact, if we have a "reference implementation" is because the thruth is in the source code instead of being in a specification document and that's also a problem. Moreover, you need to be a top hacker to understand the reference implementation source code because it is a bit messy so, if I am right, the thruth is in a messy piece of code.

These facts don't look very well because it means centralization and unilateral control. And there are other signals here and there, in this moment I can read "News: Bitcoin Core 0.9.3 has been released. Download." in the this forum header. I mean, it is more and more like the "official" node. Oh, the alert message described in the protocol specs looks like a centralised broadcast system (yes, I understand it is okay and is there for a good reason)

I think something should be done in order to improve this situation. I know it sounds awful but we need an agreement system like a IETF for the bitcoin protocol.

Is something similar in the roadmap?
1711726342
Hero Member
*
Offline Offline

Posts: 1711726342

View Profile Personal Message (Offline)

Ignore
1711726342
Reply with quote  #2

1711726342
Report to moderator
It is a common myth that Bitcoin is ruled by a majority of miners. This is not true. Bitcoin miners "vote" on the ordering of transactions, but that's all they do. They can't vote to change the network rules.
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction.
cbeast
Donator
Legendary
*
Offline Offline

Activity: 1736
Merit: 1006

Let's talk governance, lipstick, and pigs.


View Profile
November 28, 2014, 05:57:54 AM
 #2

You must be new here. These questions gets asked about once per week. I'm probably not the best one to answer these, but I'll give it  shot.

Hi,

Everytime I tell some people about bitcoin they ask me "who controls it?". I tell them nobody does it because it is a protocol just like http or smtp. However sometimes I think that that is not true. Of course nobody controls the currency but the bitcoin core team is who takes the decisions about the "reference" implementation, what means it is the team that update the protocol even when there are other full-node implementations out there.
The core team leads the development, but doesn't dictate adoption. If fact, they encourage people to not use new updates unless they are a bug fix, until they are confident it is useful to them. Miners themselves make the choice and some are still using older versions of the protocol. If an implementation is not popular, it will be dropped.

In fact, if we have a "reference implementation" is because the thruth is in the source code instead of being in a specification document and that's also a problem. Moreover, you need to be a top hacker to understand the reference implementation source code because it is a bit messy so, if I am right, the thruth is in a messy piece of code.
Basically you're saying that "math is hard" and I won't disagree. Computers are complex, but I use them. Cars too. There are many versions, I can choose the one I want based on my needs. Bitcoin is holding up against thousands of competitors because it's that good.

These facts don't look very well because it means centralization and unilateral control. And there are other signals here and there, in this moment I can read "News: Bitcoin Core 0.9.3 has been released. Download." in the this forum header. I mean, it is more and more like the "official" node. Oh, the alert message described in the protocol specs looks like a centralised broadcast system (yes, I understand it is okay and is there for a good reason)

I think something should be done in order to improve this situation. I know it sounds awful but we need an agreement system like a IETF for the bitcoin protocol.

Is something similar in the roadmap?
We don't really need the Bitcoin alert system at all, it's just a convenience. We need better watchdogs with more powerful analytic tools. They can develop their own alert systems.

Any significantly advanced cryptocurrency is indistinguishable from Ponzi Tulips.
lontivero (OP)
Full Member
***
Offline Offline

Activity: 164
Merit: 126

Amazing times are coming


View Profile
November 28, 2014, 06:11:21 AM
 #3

Thank you, @cbeast. Yes, I am new in the development sub-forum and I didn't know this is a common question ;(

Is some people doing something about it?

doge94
Sr. Member
****
Offline Offline

Activity: 349
Merit: 250


View Profile
November 29, 2014, 10:34:22 PM
 #4

The issue in creating a standard protocol is documenting all the bugs in the current reference implementation. Until this happens most other implementations will fork the chain.
gmaxwell
Moderator
Legendary
*
expert
Offline Offline

Activity: 4158
Merit: 8343



View Profile WWW
November 30, 2014, 12:59:59 AM
 #5

The issue in creating a standard protocol is documenting all the bugs in the current reference implementation. Until this happens most other implementations will fork the chain.
No, they'll continue to fork the chain regardless; many (most?) of the previously observed implementation behaviour discrepancies have been in clearly well documented behaviour; or even in behaviour triggered on the conformance testing harness. Turns out that getting radically different software to behave in a manner which is absolutely identical under all conditions is quite hard.
BlindMayorBitcorn
Legendary
*
Offline Offline

Activity: 1260
Merit: 1115



View Profile
November 30, 2014, 01:10:01 AM
Last edit: November 30, 2014, 04:28:24 AM by BlindMayorBitcorn
 #6

You must be new here. These questions gets asked about once per week. I'm probably not the best one to answer these


I disagree.

You, sir, are true gentlemen.

Forgive my petulance and oft-times, I fear, ill-founded criticisms, and forgive me that I have, by this time, made your eyes and head ache with my long letter. But I cannot forgo hastily the pleasure and pride of thus conversing with you.
coldcryptos
Newbie
*
Offline Offline

Activity: 6
Merit: 0


View Profile WWW
December 01, 2014, 05:30:21 PM
 #7

I'll take one perfect world, please.
cbeast
Donator
Legendary
*
Offline Offline

Activity: 1736
Merit: 1006

Let's talk governance, lipstick, and pigs.


View Profile
December 01, 2014, 11:39:13 PM
 #8

I'll take one perfect world, please.
Hold that thought.

Any significantly advanced cryptocurrency is indistinguishable from Ponzi Tulips.
altcoinex
Sr. Member
****
Offline Offline

Activity: 293
Merit: 250


Director - www.cubeform.io


View Profile WWW
December 06, 2014, 06:19:34 PM
 #9

I think cbeast covers this topic very well, but I wanted to add a Consideration:

In fact, if we have a "reference implementation" is because the thruth is in the source code instead of being in a specification document and that's also a problem. Moreover, you need to be a top hacker to understand the reference implementation source code because it is a bit messy so, if I am right, the thruth is in a messy piece of code.

It is my opinion that this statement is inaccurate. While it's development path has left it as -- code that by many programming standards might be 'messy' -- It is very far from being unreadable. I think the reputation for it being such a disaster has been greatly exaggerated for specific effect in a few interviews, leaving those unfamiliar with it to assume it far worse off than it actually is. Any developer with basic knowledge of software engineering, c++, and cryptography concepts are all that is required to read and understand the vast majority of the reference code. The few advanced programming concepts implemented in the reference client or design plans typically have clear explanation available, and while may take a moment to fully grasp are well within reason for an average developer. I think it is important to not let the rumor of bitcoin's code being 'a mess' demotivate people who would otherwise be interested in it's development..



                                     ╓╢╬╣╣╖
                                   ┌║██████║∩
                                   ]█████████
                                    ╜██████╝`
                                      ╙╜╜╜`
                                   ╓╥@@@@@@╥╓
         ╓╖@@╖,                 ,@║██████████╢@,                 ,╓@@╖╓
       ╓╢██████╢.              ╓╢███████████████╖               ║╢█████║╓
       ║█████████    ,,╓╓,,   ┌║█████████████████┐   ,,╓╓,,    ]█████████
       └╢██████║` ╓╢║██████╢║∩``╙╙╙╙╙╙╙╙╙╙╙╙╙╙╙╙╙`»╢╢██████╢║╖  ║███████╜
         "╜╜╜╜` ╖╢█████████╣╜                      └╢██████████@ `╜╜╜╜╜
               ║██████████╜                          ╙╢██████████
              ┌█████████╜                              ╙╢█████████
              └███████╨`                                 ╜████████
               ║████╨╜                                    `╢█████
                ╙╢╣╜                                        └╢█╜
                ,,                                            ,,
             ╓@║██┐                                          ┌██║@╓
            ╢██████                                          ]█████H
           ╢███████∩                                        ┌████████
  ╓@@@@╓   █████████                                        ║████████`  ╓@@@@╖
╓╢██████║. █████████∩                                      ┌█████████ ,║███████╖
██████████ └█████████                                      ██████████ ]█████████
`║██████╜`  └╢████████                                    ┌███████╣╜   ╙██████╨`
  `╙╜╜╙`      `╙╨╢████                                    █████╝╜`       `╙╜╜`
                      ]@╓                              ╓╖H
                      ███╢║@╓,                    ,╓@╢╢███`
                      ████████╢@╖╓.           ╓╖@║████████`
                      ]███████████╢║@╓,  ,╓@╢╢████████████
                       ╙╢█████████████╨` ╜██████████████╜
                         ╙╝╢███████║╜`    `╜║████████╝╜`
                     ,╓@@@╓  `²╙``             `╙²`  ╓@@@╖,
                    ║╢█████╢H                      ╓╢██████H
                    █████████                      █████████`
                    ╙╢██████╜                      ╙╢██████╜
                      └╨╩╝┘                          └╨╩╝╜
WINFLOW.
██
██
██
██
██
██
██
██
██
██
██
██
██
..
██
██
██
██
██
██
██
██
██
██
██
██
██
.
CIYAM
Legendary
*
Offline Offline

Activity: 1890
Merit: 1072


Ian Knowles - CIYAM Lead Developer


View Profile WWW
December 06, 2014, 06:23:18 PM
 #10

The issue that @gmaxwell has mentioned is that any alternative implementation that fails to do something that the current implementation does (which may have never been even witnessed before) would result in a fork.

So the unfortunate situation is that Bitcoin is not a protocol like HTTP but instead is the behaviour of a specific C++ program (who's every currently unknown *quirk* would have to be replicated in any other implementation).

The only way you could create a Bitcoin equivalent in another language would be to have something that behaves identically at the binary level (and that would most likely only perform slower than Bitcoin does).

With CIYAM anyone can create 100% generated C++ web applications in literally minutes.

GPG Public Key | 1ciyam3htJit1feGa26p2wQ4aw6KFTejU
gmaxwell
Moderator
Legendary
*
expert
Offline Offline

Activity: 4158
Merit: 8343



View Profile WWW
December 06, 2014, 07:31:12 PM
 #11

One of my longer term hopes around the refactoring of Bitcoin core into a separate library for consensus is that it allows us to compile the consensus parts into a simple bytecode with a narrow interface which can be executed by an easily implemented and testable virtual machine, so every implementation can just use the same bytecode and be confident that they'll be consistent.  Considering how hard it has been to get people to understand the unusual requirements of consensus, it may turn out to be hard to get people to accept the performance hit (and to avoid doing crazy JIT stuff which breaks the obvious-correctness of the approach).
lontivero (OP)
Full Member
***
Offline Offline

Activity: 164
Merit: 126

Amazing times are coming


View Profile
December 06, 2014, 10:07:13 PM
 #12

The issue that @gmaxwell has mentioned is that any alternative implementation that fails to do something that the current implementation does (which may have never been even witnessed before) would result in a fork.

So the unfortunate situation is that Bitcoin is not a protocol like HTTP but instead is the behaviour of a specific C++ program (who's every currently unknown *quirk* would have to be replicated in any other implementation).

The only way you could create a Bitcoin equivalent in another language would be to have something that behaves identically at the binary level (and that would most likely only perform slower than Bitcoin does).


That's exactly what I mean.

I've heard people saying that one of the developer's perversion is that they want to rewrite all in another programming language, everybody wants to have a library available for being used in their favorite language. It is easy to understand why devs want to use a confortable language for their projects so, even when that could be too hard in the case of bitcoin full node, new implementations are going to appear and that could be a problem.

It is clear that bitcon is not a commnon thing and there are lost and lots of details to have in mind and tests (like security). However, if I am rigth and new implementations are developed then the bitcoin core team will need to do something.
One of my longer term hopes around the refactoring of Bitcoin core into a separate library for consensus is that it allows us to compile the consensus parts into a simple bytecode with a narrow interface which can be executed by an easily implemented and testable virtual machine, so every implementation can just use the same bytecode and be confident that they'll be consistent.  Considering how hard it has been to get people to understand the unusual requirements of consensus, it may turn out to be hard to get people to accept the performance hit (and to avoid doing crazy JIT stuff which breaks the obvious-correctness of the approach).

 

I think that separate library (or specialized libraries) is an very good idea. I don't understand why a VM could be required.   
CIYAM
Legendary
*
Offline Offline

Activity: 1890
Merit: 1072


Ian Knowles - CIYAM Lead Developer


View Profile WWW
December 07, 2014, 05:13:07 PM
 #13

I think that separate library (or specialized libraries) is an very good idea. I don't understand why a VM could be required.   

I agree with this - Linux has had a long history of libs written in C that end up being available for other languages - why the need to go against that and create a Java thing (as I assume is being suggested)?

With CIYAM anyone can create 100% generated C++ web applications in literally minutes.

GPG Public Key | 1ciyam3htJit1feGa26p2wQ4aw6KFTejU
TierNolan
Legendary
*
Offline Offline

Activity: 1232
Merit: 1077


View Profile
December 08, 2014, 12:21:30 AM
 #14

One of my longer term hopes around the refactoring of Bitcoin core into a separate library for consensus is that it allows us to compile the consensus parts into a simple bytecode with a narrow interface which can be executed by an easily implemented and testable virtual machine, so every implementation can just use the same bytecode and be confident that they'll be consistent.  Considering how hard it has been to get people to understand the unusual requirements of consensus, it may turn out to be hard to get people to accept the performance hit (and to avoid doing crazy JIT stuff which breaks the obvious-correctness of the approach).

This VM would need to be able to have enough memory for the UTXO set.  All the crypt functions would presumably be opcodes.

Accepting blocks into the memory pool presumably doesn't need to be handled by the VM, since that won't cause a fork.  Each miner would check their own blocks before mining.

With UTXO set commitments, block verification could be more self-contained.  The VM would run on a block + Merkle path based metadata + the header chain.

1LxbG5cKXzTwZg9mjL3gaRE835uNQEteWF
erik777
Sr. Member
****
Offline Offline

Activity: 504
Merit: 250


Earn with impressio.io


View Profile
December 08, 2014, 01:13:42 AM
 #15

I agree that it would be a good idea to define a language agnostic spec for the communications protocol, and I haven't heard a single valid argument against it.  If you are against defining an open standard, at least be honest and just say no one is motivated to do it or share whatever concern you have would come out of such an effort.  Honestly is ok.  But, obfuscating lack of motivation with "the code is OK" is just a distraction that undermines the value others perceive.  Regardless of whether or not someone currently seasoned in C++ finds it OK the way it is, an open specification would provide value to those of us not so motivated to pick apart a C++ program just to understand the communications going over the wire.  This could also enhance peer review by increasing the respective peer review audience, and possibly foster more innovation around the network.    

EDIT: The next thing I read after posting this happened to be an excellent real example of the value of better peer review of the protocol:

https://bitcointalk.org/index.php?topic=359582.0

Can someone tell me what a VM means in this conversation?  I certainly doesn't sound like a virtual machine that runs an OS, nor does it sound like a JVM, unless I'm just not connecting the dots.    


  

.▄███     ██████     ███▄
██████   ███████   ██████
 ██████ ██████████ ██████
  ██████████████████████
   █████████  ████████
    ██████    ██████
    ███████    ██████
   █████████  █████████
  ██████████████████████
 ██████ ██████████ ██████
██████   ██████   ██████
 ▀███     ██████     ███▀
IMPRESSIO     ▄███████████████▄
     ██             ██
     ▀███████████████▀
           ██ ██
           ██ ██
       ▄▄█████████▄▄ ▄███▄
    ▄███▀▀       ▀▀████ ▀██▄
  ▄██▀   ▄▄█████▄▄   ▀██▄ ██
 ▄██  ▄███  █  █████▄  ██▄█▀
 ██  ███         █████  ██
██  ██████  ███   █████  ██
██  ██████  ▀▀▀  ▄█████  ██
██  ██████  ▄▄▄▄  █████  ██
██  ██████  ████   ████  ██
 ██  ███          ████  ██
 ▀██  ▀███  █  █████▀  ██▀
  ▀██▄   ▀▀█████▀▀   ▄██▀
    ▀███▄▄       ▄▄███▀
       ▀▀█████████▀▀
gmaxwell
Moderator
Legendary
*
expert
Offline Offline

Activity: 4158
Merit: 8343



View Profile WWW
December 09, 2014, 02:21:09 AM
Last edit: December 09, 2014, 09:33:00 AM by gmaxwell
 #16

I agree that it would be a good idea to define a language agnostic spec for the communications protocol, and I haven't heard a single valid argument against it.
The communications are already speced out in reasonable enough detail that multiple people have implemented talking to it from just the documentation. (e.g. from he developer guide and Bitcoin Wiki/

Quote
If you are against defining an open standard, at least be honest and just say no one is motivated to do it or share whatever concern you have would come out of such an effort.  Honestly is ok.
You're accusing people of being dishonest?  Really?  Try engaging in a little professionalism. Next time someone gives you an explanation you don't understand, you could try not accusing them as dishonesty as a first step. It certainly doesn't motivate people to spend their time educating you on a subject about which you are clearly ill-informed when you take that approach.

Quote
the value others perceive
Sometimes people perceive value incorrectly. It's not that the code is "fine" its that for the normative operation of a consensus system it's all that actually matters. Additional informative documentation is useful too, and people have written a fair amount of it.

Quote
EDIT: The next thing I read after posting this happened to be an excellent real example of the value of better peer review of the protocol:
Funny, I don't see how thats an example of better peer review of the protocol at all: It requires only an understanding of the system at the level in the original Bitcoin whitepaper (which has been available since 2009). OTOH, I've used that work before as an example of how academia needs more peer review from industry-- because that design breaks decentralization which is the whole point of Bitcoin, and Amiller proposed a very similar design (and arguably better, since there were some additional attacks it resisted) back in 2011 which the work should have cited.

Quote
Can someone tell me what a VM means in this conversation?  I certainly doesn't sound like a virtual machine that runs an OS, nor does it sound like a JVM, unless I'm just not connecting the dots.
A bytecode simulator, like the JVM. One intentionally constructed to be very simple in order to avoid the extreme reimplementation risk in consensus code.
gmaxwell
Moderator
Legendary
*
expert
Offline Offline

Activity: 4158
Merit: 8343



View Profile WWW
December 09, 2014, 02:25:14 AM
 #17

This VM would need to be able to have enough memory for the UTXO set.  All the crypt functions would presumably be opcodes.
It wouldn't: You'd use a authenticated data structure so that all the input could come from an unverified domain. It could fail completely but it would know it... even a not found would be expected to prove non-membership by providing fragments for a search query that failed against the current root.  The UTXO doesn't even need to be committed in the blockchain for this to work, e.g. the bytecode machine could just keep track of the root itself.

(If you wanted to be especially paranoid you could use path-oram to make sure that there was no visible access pattern or data sequence outside of the sandbox thta was common on multiple hosts which could cause a systemic failure).
hhanh00
Sr. Member
****
Offline Offline

Activity: 467
Merit: 266


View Profile
December 09, 2014, 03:00:12 AM
 #18

I agree that it would be a good idea to define a language agnostic spec for the communications protocol, and I haven't heard a single valid argument against it.
The communications are already speced out in reasonable enough detail that multiple people have implemented talking to it from just the documentation. (e.g. from he developer guide and Bitcoin Wiki/
I can attest that it's possible to write a client without looking at the reference code. @erik777 - what do you think a standardization process would add? Bear in mind that HTML 5 doesn't have an official spec and is still used by most of the internet.

erik777
Sr. Member
****
Offline Offline

Activity: 504
Merit: 250


Earn with impressio.io


View Profile
December 09, 2014, 05:28:20 AM
 #19

gmaxwell, i was replying to one guy's statement, but didn't want to be too obvious by quoting him.  here's what he said that I didn't think the OP deserved:

"Any developer with basic knowledge of software engineering, c++, and cryptography concepts are all that is required to read and understand the vast majority of the reference code."

The documentation you linked to, on the other hand, is very useful, and reads a lot like a spec. 

Now that you pointed out the VM definition here, that is ironically what I have been thinking of developing for years for P2P, but with a broader context, a re-usable a service layer.  I'd like to create a P2P platform that can be used to create all kinds of novel things. 

I like the core functionality that retroshare has.  but, unfortunately, it's not spec'd to be a VM'd P2P platform, although they have tried to make it plug-able and have built good re-usable functionality like its cache services.  I'd like to help build a platform that can provide vast capabilities -- building blocks that can be used to create higher level craziness limited only by imagination.  A VM could provide standard access to the building blocks and simplify development of p2p apps, while protecting the machine running it.  To promote rapid adoption, I'd like a reward system for those running nodes dedicating resources, with perhaps a coin dedicated completely to machine to machine task interchange and resource sharing.   


.▄███     ██████     ███▄
██████   ███████   ██████
 ██████ ██████████ ██████
  ██████████████████████
   █████████  ████████
    ██████    ██████
    ███████    ██████
   █████████  █████████
  ██████████████████████
 ██████ ██████████ ██████
██████   ██████   ██████
 ▀███     ██████     ███▀
IMPRESSIO     ▄███████████████▄
     ██             ██
     ▀███████████████▀
           ██ ██
           ██ ██
       ▄▄█████████▄▄ ▄███▄
    ▄███▀▀       ▀▀████ ▀██▄
  ▄██▀   ▄▄█████▄▄   ▀██▄ ██
 ▄██  ▄███  █  █████▄  ██▄█▀
 ██  ███         █████  ██
██  ██████  ███   █████  ██
██  ██████  ▀▀▀  ▄█████  ██
██  ██████  ▄▄▄▄  █████  ██
██  ██████  ████   ████  ██
 ██  ███          ████  ██
 ▀██  ▀███  █  █████▀  ██▀
  ▀██▄   ▀▀█████▀▀   ▄██▀
    ▀███▄▄       ▄▄███▀
       ▀▀█████████▀▀
erik777
Sr. Member
****
Offline Offline

Activity: 504
Merit: 250


Earn with impressio.io


View Profile
December 09, 2014, 05:45:53 AM
 #20

I agree that it would be a good idea to define a language agnostic spec for the communications protocol, and I haven't heard a single valid argument against it.
The communications are already speced out in reasonable enough detail that multiple people have implemented talking to it from just the documentation. (e.g. from he developer guide and Bitcoin Wiki/
I can attest that it's possible to write a client without looking at the reference code. @erik777 - what do you think a standardization process would add? Bear in mind that HTML 5 doesn't have an official spec and is still used by most of the internet.

That's true.  Yet, good specs do mature to standardization and an open process independent of any one implementation. 

http://arstechnica.com/information-technology/2014/10/html5-specification-finalized-squabbling-over-who-writes-the-specs-continues/

Bitcoin has come a long way in 5 years.  Kudos to all the great innovators and hard laborers who made it happen.  The question is how best to objectively advance it over the next 10 years.  Should a standard drive a reference implementation, or a reference implementation drive the standard? 

That said, you could say there is some hair splitting in this discussion, as bitcoin has naturally taken on traits of an open standard through people's drive to improve implementations and the long-term health and viability of the protocol.  With open discussions, it has been behaving recently like the early days of the IETF. 


.▄███     ██████     ███▄
██████   ███████   ██████
 ██████ ██████████ ██████
  ██████████████████████
   █████████  ████████
    ██████    ██████
    ███████    ██████
   █████████  █████████
  ██████████████████████
 ██████ ██████████ ██████
██████   ██████   ██████
 ▀███     ██████     ███▀
IMPRESSIO     ▄███████████████▄
     ██             ██
     ▀███████████████▀
           ██ ██
           ██ ██
       ▄▄█████████▄▄ ▄███▄
    ▄███▀▀       ▀▀████ ▀██▄
  ▄██▀   ▄▄█████▄▄   ▀██▄ ██
 ▄██  ▄███  █  █████▄  ██▄█▀
 ██  ███         █████  ██
██  ██████  ███   █████  ██
██  ██████  ▀▀▀  ▄█████  ██
██  ██████  ▄▄▄▄  █████  ██
██  ██████  ████   ████  ██
 ██  ███          ████  ██
 ▀██  ▀███  █  █████▀  ██▀
  ▀██▄   ▀▀█████▀▀   ▄██▀
    ▀███▄▄       ▄▄███▀
       ▀▀█████████▀▀
gmaxwell
Moderator
Legendary
*
expert
Offline Offline

Activity: 4158
Merit: 8343



View Profile WWW
December 09, 2014, 09:20:35 AM
Last edit: December 09, 2014, 10:56:43 AM by gmaxwell
 #21

Quote
The question is how best to objectively advance it over the next 10 years. Should a standard drive a reference implementation, or a reference implementation drive the standard?
As several people have pointed out, only one of these is _technically possible_ because Bitcoin is a globally consistent consensus system implemented by machines that execute implementations and not prose specifications, regardless of whatever would be considered "best" in a world of frictionless spherical cows unbounded by constraints like the possible.  (I am assuming you're using a narrow definition of "standard": An implementation is merely a infinitely precise specification constructed for the benefit of machine enactment, after all. That you present it as a choice must mean that you're defining a "standard" as something inherently too imprecise to act as a concrete embodiment itself.)

It's easy for me to suffer some frustration when people continue to just repeat "but standards are good!" without demonstrating an understanding of the specific technical and political considerations in the problem space, and also while retaining a very narrow definition of what can constitute a "standard", "open", or "transparent".

Bitcoin is broken, insecure, and worthless if the machines implementing it are not able to immediately achieve globally consistency. This is a security requirement of the application domain: If the behaviour of the systems and the specification disagree, you _must_ change to match the systems or be left behind and have your funds stolen.  It's not that I don't think standards are important-- I've been involved with the IETF for many years, and attend much of the meetings and contribute to a number of working groups myself; but Bitcoin has specific technical and social requirements that make standards documents weak (even, at times, counter productive) tools when talking about the behaviour of the established production system.  Unlike TCP, SMTP, HTTP, etc. Bitcoin doesn't simply need to be somewhat inter-operable: when it comes to consensus behaviour all of the systems that constitute the network must behave in deeply identical ways. If an attacker can expose a widespread inconsistency in behaviour, however small or burred, the system faults globally and funds can be stolen even from people whos own systems are not having problems.  If the system were not demanding of consistency Bitcoin would be insecure in other ways instead.  This is just a technically harder challenge than faced by other protocols.

There is another layer to this--  Standards process ultimately rest on the politics of mankind. Most standards efforts are remarkably malleable to politics, easily bent by monied interests to their own ends... as elegantly demonstrated by some of the backdoored cryptography thats made its way into various standards, or rent seeking patent traps that required in many other international standards. And even as we speak, various groups who have little understanding of Bitcoin technically or involvement in the ecosystem have been holding workshops to "standardize it". It may well be that the people behind these efforts are completely innocent of ill-intent (beyond, perhaps, a little vanity and self-promotion), but levers that create a point of control will be exploited in the future if they can be exploited; doubly so when the people involved do not understand the risks.

Bitcoin was created to build a system whos behaviour was more deterministic than is otherwise possible in a world where political expediency can sometimes override "constitutional" guarantees. While it's useful for people to discuss Bitcoin, and nudge it here and there, without the ability to largely elevate itself above political expediency, Bitcoin serves little purpose. I think we should all support efforts to communicate better about Bitcoin, including building better descriptive text to build an on-ramp to a more complete understanding and aid people who only need an informal view. I've contributed time to help improve the developer's guide, help publish BIPs, etc. But there is authority and autonomy risk from crossing over into the realm of proscriptive text that ultimately seeks to control the system rather than the system controlling the text. This is why, for example, the BIP process as a whole is intended to be rather descriptive and there are BIPs for a number of things that quite a few experienced engineers in the space consider to be seriously ill-advised.

Another way I could present your question is "Should a human readable but inherently ambiguous definition requiring politicized interpretation and adjustment drive the definition of Bitcoin? or should Bitcoin be defined by math... even though math is not always easy to read?"

I think the Bitcoin community is smart enough to realize this, and not be deceived by people peddling a political and inherently unrepresentative standard process "driving" Bitcoin to change out from under them and potentially against their interest as "transparency" and "openness",  when it's really is anything but... making something more readable at the cost of being well defined may not be an improvement to transparency at all. A lot of contemporary standards processes could learn a lot about openness, equality, and transparency from Bitcoin. Even inside the IETF it's not too uncommon to hear the sentiment that the IETF has somewhat lost its way, now too influenced by politics and personalities than by running code. In the context of conventional standards this may just be a source of unfortunate inefficiency and avoidable failure, but in the context of Bitcoin the risk is more fundamental and so we must step carefully.

When someone is interested in gravity we can describe it informally, but when they want or need to really understand it-- in some way that has consequences-- we don't pretend that we can shy away from precise formal descriptions-- to do so would be to undermine engineering, and it would result in people dying. We certainly don't seek to "specify" it with prose and expect the universe to obey. Nor should we with Bitcoin. Bitcoin is the physics of a virtual universe, if you will. And like actual physics there is no replacement for a precise, formal, description--- especially if you're in the business of building the machinery of that universe yourself.

I'd like to help build a platform that can provide vast capabilities
What I was talking about there isn't about plug-ability or extensibility; it's orthogonal to it and wouldn't itself offer any additional flexibility. The motivation there is being able to achieve absolute consistency in the parts of the system which MUST be absolutely consistent while still allowing for multiple implementations by reducing the surface area where things could go wrong.

The kind of flexibly you're thinking of might also benefit from some of the same tools... but a consensus system like Bitcoin has some pretty incredible overheads that preclude a good bit of application space by their very nature.
spin
Sr. Member
****
Offline Offline

Activity: 362
Merit: 258


View Profile
December 09, 2014, 10:18:34 AM
 #22

Wow, great post.  Learning much from it.  Perhaps worth perserving somewhere on the wiki?

If you liked this post buy me a beer.  Beers are quite cheap where I live!
bc1q707guwp9pc73r08jw23lvecpywtazjjk399daa
lontivero (OP)
Full Member
***
Offline Offline

Activity: 164
Merit: 126

Amazing times are coming


View Profile
December 09, 2014, 01:58:03 PM
 #23

I've been thinking about all this, about a standard, the code and the available documentation and I have to admit that there were several misunderstanding from my side.

1) Source code is the more detailed specification you can find. This is always true.

2) The reference implementation is not messy code, in fact it is pretty clear now. Last time I took a look at it was in 2011/2012.

3) Available documentation is very good. I didn't know about the developer guide before this post and it is rather detailed (the wire protocol and other low level data structures are well described in other places).

I think that my misinformation stole time to @gmaxwell and others however maybe it is a good thing because other developers could have the same questions about standards.

I'd love to have a library (a .dll) to link in my project with all (or at least the hard part) what is needed to implement a node and I am sure it is the way to move forward. If a VM is a better option I would like to understand why.

   
CIYAM
Legendary
*
Offline Offline

Activity: 1890
Merit: 1072


Ian Knowles - CIYAM Lead Developer


View Profile WWW
December 09, 2014, 03:15:25 PM
 #24

I am also keen to understand the VM suggestion and why @gmaxwelll thinks that would be better than just creating a C/C++ lib with wrappers for other languages to use.

Are we talking about a new VM that is just for Bitcoin?

With CIYAM anyone can create 100% generated C++ web applications in literally minutes.

GPG Public Key | 1ciyam3htJit1feGa26p2wQ4aw6KFTejU
theymos_away
Member
**
Offline Offline

Activity: 82
Merit: 26


View Profile
December 09, 2014, 05:16:48 PM
 #25

With the sort of VM that gmaxwell is talking about, all of Bitcoin's consensus rules would be specified in a simple and exact programming language (something like Bitcoin's Script, maybe), and then every full node would include this consensus specification verbatim in its code and directly interpret it when checking things against the consensus rules. The specification would not describe or dictate the rules -- it would literally *be* the rules. This ensures that all full nodes have exactly the same behavior, but since each implementation designs its own VM/interpreter for processing the specification, different designs for the lower-level storage operations (etc.) would be possible.

This seems to me like a very good/interesting idea.
CIYAM
Legendary
*
Offline Offline

Activity: 1890
Merit: 1072


Ian Knowles - CIYAM Lead Developer


View Profile WWW
December 09, 2014, 05:52:53 PM
 #26

I very much doubt that Bitcoin's script could be of any use as it is not Turing complete and although actual Bitcoin transactions are handled by it things like re-orgs cannot be handled by it at all (nor could it handle any of the p2p stuff).

With CIYAM anyone can create 100% generated C++ web applications in literally minutes.

GPG Public Key | 1ciyam3htJit1feGa26p2wQ4aw6KFTejU
theymos_away
Member
**
Offline Offline

Activity: 82
Merit: 26


View Profile
December 09, 2014, 06:01:42 PM
 #27

I very much doubt that Bitcoin's script could be of any use as it is not Turing complete and although actual Bitcoin transactions are handled by it things like re-orgs cannot be handled by it at all (nor could it handle any of the p2p stuff).


That's why I said "something like". Script would indeed be completely unsuitable. But the specification language would have a similar purpose, and maybe a similarly-simple stack-based language would be appropriate. (I don't know much about how something like this should actually be designed, though.)
CIYAM
Legendary
*
Offline Offline

Activity: 1890
Merit: 1072


Ian Knowles - CIYAM Lead Developer


View Profile WWW
December 09, 2014, 06:33:56 PM
Last edit: December 09, 2014, 06:45:38 PM by CIYAM
 #28

Quote from: theymos_away link=topic=876020.msg9788585#msg9788585 date=1418148102s
... maybe a similarly-simple stack-based language would be appropriate....

Personally I would hope not - I was involved in a hugely funded project to build a stack based VM system before Java existed in the early 90's and as a programmer (who worked on that project) I can say that I am no fan of such stack based languages (they are far less elegant than using a more traditional CPU style approach).

With CIYAM anyone can create 100% generated C++ web applications in literally minutes.

GPG Public Key | 1ciyam3htJit1feGa26p2wQ4aw6KFTejU
2112
Legendary
*
Offline Offline

Activity: 2128
Merit: 1060



View Profile
December 09, 2014, 07:30:15 PM
 #29

Are we talking about a new VM that is just for Bitcoin?
It wouldn't be "just for Bitcoin", but would be "where Bitcoin-related programs are particuarly short". I would presume that it would have (at least some) 256-bit registers and the elliptic curve operations would be single instructions.

For the similar projects please skim through the zk-SNARK paper to get a high-level overview of the Tiny-RAM machine (and related GNU C compiler back-end) implemented there.

https://eprint.iacr.org/2013/507.pdf

The goals are also somewhat similar to the goals of SystemC, a C++ subset/superset.

http://en.wikipedia.org/wiki/SystemC

The main objective is that the architecture must be so precisely defined as to allow automatic synthesis of equivalent digital circuits and machine-generated proofs about some programs expressed in that language/bytecode.

as it is not Turing complete
It has function call operator (used in P2SH) that allows implementation of iteration through recursion, so this old chestnut should better die.

Please comment, critique, criticize or ridicule BIP 2112: https://bitcointalk.org/index.php?topic=54382.0
Long-term mining prognosis: https://bitcointalk.org/index.php?topic=91101.0
erik777
Sr. Member
****
Offline Offline

Activity: 504
Merit: 250


Earn with impressio.io


View Profile
December 10, 2014, 01:54:58 AM
 #30

Quote
The question is how best to objectively advance it over the next 10 years. Should a standard drive a reference implementation, or a reference implementation drive the standard?
As several people have pointed out, only one of these is _technically possible_ because Bitcoin is a globally consistent consensus system implemented by machines that execute implementations and not prose specifications

implementations of what?  machines that communicate in a predetermined agreed language implement a specification, regardless of how that spec is defined, unless it is just one program.  saying that a program implements a spec defines by solely by its own code isn't really an implementation of anything.   

It's easy for me to suffer some frustration when people continue to just repeat "but standards are good!" without demonstrating an understanding of the specific technical and political considerations in the problem space, and also while retaining a very narrow definition of what can constitute a "standard", "open", or "transparent".
People who are pro-standards, or anti-standards, are both extremists.  There are good standards, bad standards, and mediocre standards.  We learn and improve.  How did the good standards come about and avoid becoming bad standards?  Bitcoin is built on many standards which, if you like Bitcoin, you hopefully agree they are good standards to have, such as TCP.   

Bitcoin is broken, insecure, and worthless if the machines implementing it are not able to immediately achieve globally consistency.

True of any communications protocol.  It's a good thing HTTP works, or we wouldn't be here right now discussing this. 

This is a security requirement of the application domain: If the behaviour of the systems and the specification disagree, you _must_ change to match the systems or be left behind and have your funds stolen.  It's not that I don't think standards are important-- I've been involved with the IETF for many years, and attend much of the meetings and contribute to a number of working groups myself; but Bitcoin has specific technical and social requirements that make standards documents weak (even, at times, counter productive) tools when talking about the behaviour of the established production system.  Unlike TCP, SMTP, HTTP, etc. Bitcoin doesn't simply need to be somewhat inter-operable: when it comes to consensus behaviour all of the systems that constitute the network must behave in deeply identical ways. If an attacker can expose a widespread inconsistency in behaviour, however small or burred, the system faults globally and funds can be stolen even from people whos own systems are not having problems.  If the system were not demanding of consistency Bitcoin would be insecure in other ways instead.  This is just a technically harder challenge than faced by other protocols.

It is true there is greater security risk with bitcoin.  But, that doesn't mean that new code preceding specification is inherently more secure than having a spec to define and discuss a change beforehand.  In fact, if you really felt comfortable with the way new code is being written to create new implementations of bitcoin, you wouldn't be proposing a VM to address this concern.  To be sure, I think the VM idea is a very good one.  I'm just using it as an example because you clearly don't believe things are perfect the way they are today and the potential risk of new implementations.  If it aint broken, don't fix it, right? 

There is another layer to this--  Standards process ultimately rest on the politics of mankind....
Politics sure is a fuzzy word in any scientific conversation.  Let's reverse this.  Are you saying that open source projects don't have politics?  Are all developers always in perfect agreement.  That's why I like the early days of the IETF.  That was about as minimal on the politics as you could get.  It was engineers who worked together very well to create new things together.  It wasn't until about the mid 90s that the culture began to change.  To some extent, I think we both agree that culture is always a risk.  But, I believe that risk is just as real without a standards dialog.  As for some of your concerns, that's about WHO is influencing the standards, or what type of person, not the fact that the standardization process is visible and perhaps a bit formal.  That said, I haven't proposed any type of process.  I agree that if one were to develop a process, it might need to be a bit unique for Bitcoin to optimize the primary objectives of Bitcoin: Security, Scalability and De-centralization, with a high bar for any direct impact ability (e.g., voting) to ensure technical credibility and common ground on core principles.  Yet, the non-direct impacting open discussions could prove incredibly value-able towards Bitcoin's objectives.  And the visibility on the process cannot possibly hurt.       

Bitcoin was created to build a system whos behaviour was more deterministic than is otherwise possible in a world where political expediency can sometimes override "constitutional" guarantees. While it's useful for people to discuss Bitcoin, and nudge it here and there, without the ability to largely elevate itself above political expediency, Bitcoin serves little purpose. I think we should all support efforts to communicate better about Bitcoin, including building better descriptive text to build an on-ramp to a more complete understanding and aid people who only need an informal view. I've contributed time to help improve the developer's guide, help publish BIPs, etc. But there is authority and autonomy risk from crossing over into the realm of proscriptive text that ultimately seeks to control the system rather than the system controlling the text. This is why, for example, the BIP process as a whole is intended to be rather descriptive and there are BIPs for a number of things that quite a few experienced engineers in the space consider to be seriously ill-advised.

+1 and I understand your concerns.  Keep it scientific with the core team dedicated demonstrating technical excellence and a common vision.  Note, the latter hasn't always been clear to-date, such as with anonymity and fungibility. 

Another way I could present your question is "Should a human readable but inherently ambiguous definition requiring politicized interpretation and adjustment drive the definition of Bitcoin? or should Bitcoin be defined by math... even though math is not always easy to read?"
Like SSL and PKI? 

You again make it sound ilke a choice between math and politics.  Math and politics will always be there.  The only question is HOW you manage the politics.  Do you hide it and pretend it's not there, because there is so much math?  Or do you limit it in your core charter with checks and balances and hopefully an agreement on guiding principles?     

... in the context of Bitcoin the risk is more fundamental and so we must step carefully.

True, except the "..." part.  hahaha 

When someone is interested in gravity we can describe it informally, but when they want or need to really understand it-- in some way that has consequences-- we don't pretend that we can shy away from precise formal descriptions-- to do so would be to undermine engineering, and it would result in people dying.

ROFL!  Not sure how you think Bitcoin is as complex as physics computations, or how you think people will die if the current political status quo has increased visibility.

We certainly don't seek to "specify" it with prose and expect the universe to obey. Nor should we with Bitcoin. Bitcoin is the physics of a virtual universe, if you will. And like actual physics there is no replacement for a precise, formal, description--- especially if you're in the business of building the machinery of that universe yourself.

Except Bitcoin isn't a virtual universe. 

I'd like to help build a platform that can provide vast capabilities
What I was talking about there isn't about plug-ability or extensibility; it's orthogonal to it and wouldn't itself offer any additional flexibility. The motivation there is being able to achieve absolute consistency in the parts of the system which MUST be absolutely consistent while still allowing for multiple implementations by reducing the surface area where things could go wrong.

The kind of flexibly you're thinking of might also benefit from some of the same tools... but a consensus system like Bitcoin has some pretty incredible overheads that preclude a good bit of application space by their very nature.

I know.  I was digressing a bit into a vision that in itself has nothing to do with Bitcoin.  I do see a new M2M virtual currency being required to grease the wheels of trustless resource and task distribution, developed from the ground up to meet the needs of M2M.  Examples of how requirements would differ from Bitcoin: transactions would be incredibly micro.  Yet, because these are virtual machines in a virtual universe, they could settle debt in much larger time spans, such as once a day, decreasing the load on the primary order book.  It could be a combination of trusted and trustless, as well. 

I appreciate your passion for the success of Bitcoin.  You do bring a lot of insight and I do empathize with your concerns even when we don't totally agree on whether or not the current political status quo is enough to ensure the success of Bitcoin over the next 10 years.     

.▄███     ██████     ███▄
██████   ███████   ██████
 ██████ ██████████ ██████
  ██████████████████████
   █████████  ████████
    ██████    ██████
    ███████    ██████
   █████████  █████████
  ██████████████████████
 ██████ ██████████ ██████
██████   ██████   ██████
 ▀███     ██████     ███▀
IMPRESSIO     ▄███████████████▄
     ██             ██
     ▀███████████████▀
           ██ ██
           ██ ██
       ▄▄█████████▄▄ ▄███▄
    ▄███▀▀       ▀▀████ ▀██▄
  ▄██▀   ▄▄█████▄▄   ▀██▄ ██
 ▄██  ▄███  █  █████▄  ██▄█▀
 ██  ███         █████  ██
██  ██████  ███   █████  ██
██  ██████  ▀▀▀  ▄█████  ██
██  ██████  ▄▄▄▄  █████  ██
██  ██████  ████   ████  ██
 ██  ███          ████  ██
 ▀██  ▀███  █  █████▀  ██▀
  ▀██▄   ▀▀█████▀▀   ▄██▀
    ▀███▄▄       ▄▄███▀
       ▀▀█████████▀▀
Pages: 1 2 [All]
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!