strip the reference client to a border router

[grau]

Would you welcome a fork of satoshi aimed to strip it of all but the consensus algorithm, that is without wallet and the RPC replaced with a bus?

Motivation
I recommend using the 'reference' client as border router to the P2P network in production environments, then connect the BOP server inside your company to it, to get  a convenient API to Bitcoin in form of a message bus that serves applications and client side wallets.

I guess that above suggestion is not unique to BOP but is how those operate who developed significant added value around satoshi client and wonder if this use case should be the priority for reference client development instead of further extending the feature set of that fragile and crucial code base.

[gmaxwell]

Wouldn't this increase the amount of software which must be reviewed, tested, patched, and otherwise maintained? Is this just for the sake of removing parts which are largely inactive and harmless when not used?

I often don't understand proposals stated in terms of mechanisms rather than goals. Perhaps the benefits are apparent to those who would want it, but for my sake what benefit(s) would this serve which would justify the costs?

There is already have code to turn off the wallet. If there is a real application for turning off other things I don't see a reason that a fork would be required.

[grau]

The goal is to
  1. better support for feature development without touching or re-implementing functions of the reference client.
  2. reduce number of lines of code and complexity in reference client to increase stability and scaleability.

Although wallet or RPC is harmless at runtime if not used, I do believe that every line of extra code is harmful at development as it makes code review and refactoring more complex and risky.

To give a practical example:
I think payment protocol is much needed, but I wish it could be implemented without touching the code base running the consensus and it would be implemented by other than Gavin.  His in depth knowledge of the Satoshi code would be better spent on DoS protection and scaleability issues. With a stripped down reference client and a better API that development could be much better outsourced to the community.

[gmaxwell]

If Gavin was interested in (/thought his time was best spent on) those other things he would spent them on them. You can't dictate how Gavin spends his time by creating a fork.

As an aside, though the code ends up in the same binary, as I recall the payment protocol didn't touch consensus mechanism stuff, it was largely self contained and nowhere near the core stuff. Increased modularity is a long term goal to making review easier, though it's currently come a long way from the original code that freely intermixed the GUI wallet stuff with the consensus code.

[gweedo]

https://github.com/jgarzik/pynode this is kinda what you are talking about, this would be better suited than stripping out things from the reference client, which would be the hacky way to do this.

[grau]

If Gavin was interested in (/thought his time was best spent on) those other things he would spent them on them. You can't dictate how Gavin spends his time by creating a fork.

I certainly can't or want to dictate what he does or doesn't. We could however lower the barrier for those who could do stuff that does not require in depth knowledge of the real core, so he gets more help and can consider if that is where he want to be (too).

As an aside, though the code ends up in the same binary, as I recall the payment protocol didn't touch consensus mechanism stuff, it was largely self contained and nowhere near the core stuff. Increased modularity is a long term goal to making review easier, though it's currently come a long way from the original code that freely intermixed the GUI wallet stuff with the consensus code.

Even if it is better separated than older cross-cutting concepts, would't be nice if it was built on top of an API isolating the consensus layer and be in a separate project?

We agree that modularization is a goal, my question is if we should we tackle it with highest priority.

[grau]

https://github.com/jgarzik/pynode this is kinda what you are talking about, this would be better suited than stripping out things from the reference client, which would be the hacky way to do this.

Thinks I have that kinda stuff too, check my signature.

The question is if we want more that kind of stuff or rather make satoshi welcoming to extensions.

[Mike Hearn]

The Satoshi client does have a kind of internal API, if you look at how the wallet and GUI interacts with the rest of the code then you can see it. It's not perfect but polishing it hardly seems the highest priority.

I think you're right that lots of people use their software connected to a trusted Satoshi node that they run themselves, but realistically the wallet and GUI aren't just going to be deleted from the core Bitcoin code - people use them!

[grau]

The Satoshi client does have a kind of internal API, if you look at how the wallet and GUI interacts with the rest of the code then you can see it. It's not perfect but polishing it hardly seems the highest priority.

I think you're right that lots of people use their software connected to a trusted Satoshi node that they run themselves, but realistically the wallet and GUI aren't just going to be deleted from the core Bitcoin code - people use them!

I wish for an isolation much higher than a more or less defined set of function signatures and structures within the same code base.
Such isolation was not a goal for Satoshi and is apparently not a priority for the core team.

I suggested such a fork (of code) to reduce the pain of code reviews and increase the pace of innovation.
It would come at a cost. Do you think it is feasible to branch and regularly rebase such a code subset for a "border router" ?

[jgarzik]

A branch would increase work required, rather than decrease.

Here is the current rough plan:

The first step is basically complete:  a runtime no-wallet mode:  https://github.com/bitcoin/bitcoin/pull/2901    Later on, this may be extended to #ifdef'ing out wallet code, once the "no wallet" runtime code has been proven to work.

Next, headers-first sync, which fixes many issues with the bitcoin block download process in general: https://github.com/sipa/bitcoin/tree/headersfirst

Once these features in place, it becomes feasible to have the wallet client operate in a separate process from the public blockchain engine.

At that point you have a stripped down "blockchain engine" (border router) that focuses on mesh networking and accurately maintaining the chain.

[grau]

This sounds promising Jeff.  One could think we face similar challenges

A chance of such refactoring could make me picking up C++ again ... after 12 years of not missing it.

[justusranvier]

It would be cool if the Satoshi client, btcd, and BoP could all use the same APIs to talk to the various components such that the components become interchangeable.

[jl2012]

I like this idea. I use Armory only. It is way more secure than the reference client. A fully modularized system is easier to audit, maintain, and expand.

[piotr_n]

Guys, you are talking to a wall here.

I dare to state (and I am dying to be proven wrong) that making a wallet-less reference node, with minimal external dependencies will never happen - at least not made by this team.
It just isn't on their employers' agenda, despite of all the rumors and gossips that you might have heard.

The only way to have such a piece of software, is by forking the existing bitcoind and stripping it from all the crap, without even looking at the corrupt bitcoin elite, nor letting them to waste your time. If you want to have such a piece of software, then trust me: sooner you will make it yourself. I'd help you, but I prefer to raise my own child instead

[justusranvier]

The only way to have such a piece of software, is by forking the existing bitcoind and stripping it from all the crap, without even looking at the corrupt bitcoin elite, nor letting them to waste your time. If you want to have such a piece of software, then trust me: sooner you will make it yourself. I'd help you, but I prefer to raise my own child instead

Conformal Systems, and bits of proof, and Amir Taaki have already done that.

[piotr_n]

The only way to have such a piece of software, is by forking the existing bitcoind and stripping it from all the crap, without even looking at the corrupt bitcoin elite, nor letting them to waste your time. If you want to have such a piece of software, then trust me: sooner you will make it yourself. I'd help you, but I prefer to raise my own child instead

Conformal Systems, and bits of proof, and Amir Taaki have already done that.

Thanks for being so well informed, but I have also done this, so please mind not letting my fine project behind
It has everything a bitcoin node needs and performs just great; I can challenge any of the software you've listed.
And if anyone needs a mining API, just let me know - shouldn't be hard to add.

[jgarzik]

I dare to state (and I am dying to be proven wrong) that making a wallet-less reference node, with minimal external dependencies will never happen - at least not made by this team.
It just isn't on their employers' agenda, despite of all the rumors and gossips that you might have heard.

no-wallet mode will be in version 0.9.

[piotr_n]

I dare to state (and I am dying to be proven wrong) that making a wallet-less reference node, with minimal external dependencies will never happen - at least not made by this team.
It just isn't on their employers' agenda, despite of all the rumors and gossips that you might have heard.

no-wallet mode will be in version 0.9.

but from I understand, it will only be controlled by a command line switch.
all the crap, plus tones of a new crap, will be still linked with the node, won't it?

so that is definitely not what I meant by "wallet-less reference node, with minimal external dependencies".
command line switches disabling functionality do not make a code easier to audit, nor less open for a potential exploits.

and BTW man, thanks for being one of a few out there who at least try to do it the right way
https://github.com/bitcoin/bitcoin/pull/2901

[jl2012]

Is it possible to build a "no dependency" client? Just integrate all (and only) necessary external codes, e.g. SSL, so people can build it without depending on any package.

[wumpus]

Is it possible to build a "no dependency" client? Just integrate all (and only) necessary external codes, e.g. SSL, so people can build it without depending on any package.

You could use sipa's secp256 library instead of OpenSSL, that makes the blockchain handling code less dependent of third parties: https://github.com/sipa/secp256k1 (as a bonus, it's faster too!)
Note: this is at your own risk, the library is far from as well-tested as OpenSSL is. Handling cryptography is very fickle and a decision to switch the main client over is not taken lightly.

We agree that modularization is a goal, my question is if we should we tackle it with highest priority.

That's up to you! Everyone in the core team has their own priorities, and they may or may not overlap with yours (though we all agree that modularization is a good thing, but it's just not our pet project). You're welcome to join and patches are welcome!