ANN: Announcing code availability of the bitsofproof supernode

[grau]

I think miner will just stop running an implementation or version as they think it is marginalized in user population or if they think it will not get tracktion. The check is however cheap.

I think the proosal would increase network security against split and as a byproduct create a testing interface accross versions and implementations.

Since challenges would no longer be in the source tree compliant implementations could not just learn them since they have to reply correctly to any new challange.

[grau]

Sure block templates are a very nice tool for miners who want to run untrusted block creation code (and probably very useful for testing), but it would be sad if evolution forced miners to run several nodes at the same time.

Mining is already split to the process of block creation and POW since the second is no longer on CPU. Some will continue running both processes but I suspect more small ASICs and GPUs will trust pool operators they connect to for simplicity. Pool operators might differentiate themselves by having checks against several versions and implementations as added value service.

I think I will implement the proposal since it is useful for testing and make an instance of bitsofproof supernode accessible at bitsofproof.com for anyone to run checks against. Technical details will follow as it settles in the proposal I suggested.

Please participate in the design of this protocol extension proposal in the thread https://bitcointalk.org/index.php?topic=130327.0
I would like to refocus this thread to bitsofproof only.

[Jouke]

Sure block templates are a very nice tool for miners who want to run untrusted block creation code (and probably very useful for testing), but it would be sad if evolution forced miners to run several nodes at the same time.

Mining is already split to the process of block creation and POW since the second is no longer on CPU. Some will continue running both processes but I suspect more small ASICs and GPUs will trust pool operators they connect to for simplicity. Pool operators might differentiate themselves by having checks against several versions and implementations as added value service.

I think I will implement the proposal since it is useful for testing and make an instance of bitsofproof supernode accessible at bitsofproof.com for anyone to run checks against. Technical details will follow as it settles in the proposal I suggested.

Please participate in the design of this protocol extension proposal in the thread https://bitcointalk.org/index.php?topic=130327.0
I would like to refocus this thread to bitsofproof only.

I think that miners are a targetgroup in itself.

I have always wanted to provide bitcoinclients to other people, so they don't need to stay connected all the time and don't have to worry too much about the security of their computers. I wanted to run several instances of the satoshiclient, as I like how it encrypts the wallet and especially the powerful JSON RPC server. There is already so many software that works with the json-commands, it would be a great match to give a user access to a server and let them chose their own front end of their liking.

You already said that you wanted to incorporate the json rpc in your and I am wondering if you'll make it so that it would have access to multiple wallets as well?

[grau]

The plan is to unlock innovation and new uses by offering a robust and compliant kernel that is modular and extensible as a service.

The primary interface to the kernel will be defined as a Java interface (BCSAPI), available throgh usual ways of remote invocaton just by configuration. For debug insight I prefer implementing JMX.

JSON RPC calls of bitcoind could be implemented on top of that by those seek backward compatibility with existing tools. I may also provide some as it seems appropriate or for demonstration.

I will try hard not to make a specific use part of this code base.

Wallets, client servicing, connection to other enterprise software and whatever people will come up with should build on that BCSAPI kernel interface, preferably remote invoking it.

[2112]

The primary interface to the kernel will be defined as a Java interface (BCSAPI), available throgh usual ways of remote invocaton just by configuration. For debug insight I prefer implementing JMX.

BCSAPI == Bean Context Services API
or
BCSAPI == Business Connectivity Services API
?

[grau]

BitCoin Server API

[grau]

I promised some comparison of leveldb and relational model and have some feature updates:

Performance
bootstrap node from scratch (Merkle root and POW check and store until block 210.000 thereafter full validation of scripts)

in Minutes:
                    until block 210.000      then until most recent block
LevelDB               93                        61
Relational         3664                       297

That roughly 2.5 hours against 2.5 days.

I run this on:
Linux bitsofproof 3.2.0-31-generic #50-Ubuntu SMP Fri Sep 7 16:16:45 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux
with 4 2GHz processors 16GB RAM, Derby 10.9.1 with 400MB cache.

New features
I split the code base into

• supernode - the protocol implementation and storage engine
  
• supernode-api - a java interface that defines (not yet complete) BCSAPI and basic primitives needed in local code base of clients e.g. Hash, sign.
  
• supernode-testclient - a small java app that uses the BCSAPI to demonstrate connection and work with running server
  

The remote calls are accomplished with Java RMI using two way authentication and encryption over SSL.

An example use of securely connecting to remote bitsofproof supernode:
start the server (running on a linux server in a datacenter in Germany):
    nohup java -server -Xmx4g -jar target/supernode-0.7-SNAPSHOT.jar production relational &

ask for the balance of the bitsofproof donation account using the test client (running on my MacOS laptop at home in Hungary):
    java target/supernode-testclient-0.7-SNAPSHOT.jar -a 13xhxy21to93Lrb7d4ssZVaMnFtQVvRkSk
prints:
13xhxy21to93Lrb7d4ssZVaMnFtQVvRkSk balance: 1400000000
thats 14 BTC, retrieved in a roundtrip of 72 miliseconds.

the same momentary balance for 1dice8EMZmqKvrGE4Qc9bUFf9PX3xaYDp balance: 67391770
roundtrip time 658 seconds. (ca. 1.5 million transaction outputs retrieved on server side)

[BR0KK]

May a bitcoin noob ask what's that node for (or where can i read something about that?). I always thought a bitcoin qt (bitcoind) was a node in the network?

Is this something the "Average Joe" should deal with? ie. run it on a server themselves?

[grau]

May a bitcoin noob ask what's that node for (or where can i read something about that?). I always thought a bitcoin qt (bitcoind) was a node in the network?

Is this something the "Average Joe" should deal with? ie. run it on a server themselves?

bitcoind defines the network now as most nodes run it.

That one size will not fit all

• Average Joe will be tired of gigabytes of data and traffic and will use e.g. MultiBit or move away from PC to smartphones to use bitcoin.
• Merchants will want to explore new unique uses and connect bitcoin with their other other systems.

I build the bitsofproof supernode for the merchants, who need modularity, scalebility, connectivity.

[BR0KK]

Ok than you for clarification

So this node is for the big boys around here.

[grau]

Ok than you for clarification

So this node is for the big boys around here.

Also for those who wanna be big

[grau]

The bitsofproof supernode now features an asynchronous communication bus with corresponding API to subscribe to validated transactions, block chain events (extension, reorg) and block templates.

The bus allows submission of mined blocks and new transactions and is supported by a generic purpose JMS message broker, in my case Apollo MQ. The use of such message broker allows efficient message distribution to a large number of known counterparts, while taking care of authentication and authorization to publish or subscribe to different topics the messages are queued on.

Both server and evtl. clients connect to the bus using SSL sockets over the internet.

Also added documentation on configuration and database schema overview in:

https://github.com/bitsofproof/supernode/wiki

[grau]

The version is superseded and the file mentioned below is no longer offered

I have an alpha quality preview for those curious of the supernode but do not like dealing with java.

You find a windows installer under ftp://bitsofproof.com/supernode-0.7-SNAPSHOT-setup.exe

You can download a plain zip of executable here
ftp://bitsofproof.com/supernode.zip

It is a compiled version of the original java code to native windows 32. It does not require you to install a java runtime.

The program does not do anything directly useful since it has no wallet. Should you figure it is still useful for some purpose, do not use it yet for real. It just downloads the chain and then validates and relays transactions. For production chain it validates and Merkle roots and proof of work until block 213900, therafter full validation of scripts. It does full validation from the beginning on testnet3.

Execute the installer and choose some empty new directory as target (without spaces in the path like "Program Files").
Once installed unzipped you may execute as:

1. production net with LevelDB. Data will be stored in a sub directory called data

Code:

supernode-0.7-SNAPSHOT.exe production leveldb

2. production net with embedded relational db (derby) - data will be stored in a sub directory called derby

Code:

supernode-0.7-SNAPSHOT.exe production derby 

3. testnet3 in memdb - data remains in memory and starts over at every execution

Code:

supernode-0.7-SNAPSHOT.exe testnet3 memdb

It will ask for BCSAPI password at start. Just hit enter.

I am curious if the packaging works well on different machines and how you like it.

[2112]

I am curious if the packaging works well on different machines and how you like it.

Why is it asking for Administrator privileges? Can you post a plain archive?

[grau]

I am curious if the packaging works well on different machines and how you like it.

Why is it asking for Administrator privileges? Can you post a plain archive?

It should not. Is it because of the location you try to install it ? If not I create an other one.

[2112]

It should not. Is it because of the location you try to install it ? If not I create an other one.

It clearly shows in the manifest: requestedExecutionLevel="requireAdministrator". Please post something that 7-zip could open before installation.

[grau]

It should not. Is it because of the location you try to install it ? If not I create an other one.

It clearly shows in the manifest: requestedExecutionLevel="requireAdministrator". Please post something that 7-zip could open before installation.

I apologize that was not intentional. I uploaded a plain zip here:

ftp://bitsofproof.com/supernode.zip

[2112]

I apologize that was not intentional. I uploaded a plain zip here:

ftp://bitsofproof.com/supernode.zip

Thanks. It seems safe enough to run. So I tried "testnet3 memdb". It cannot bootstrap itself off the IRC on a machine with a real dual stack IPv4 + IPv6. I think this bug exists in the Satoshi's code too, I just start Satoshi with -noirc. I killed it because the error that keeps scrolling is "Peer connector IRC receive ERROR :Trying to reconnect too fast.". I don't want to DoS anybody's IRC servers.

[grau]

Thanks. It seems safe enough to run. So I tried "testnet3 memdb". It cannot bootstrap itself off the IRC on a machine with a real dual stack IPv4 + IPv6.

I am not sure, since I saw this occasionally and could not yet find the reason. Try re-running, you might get an other IRC server assigned. IRC discovery sucks. I can not offer a quick fix there I am afraid. 

Did you try production leveldb that uses DNS discovery ?

[2112]

I am not sure, since I saw this occasionally and could not yet find the reason. Try re-running, you might get an other IRC server assigned. IRC discovery sucks. I can not offer a quick fix there I am afraid. 

Did you try production leveldb that uses DNS discovery ?

"testnet3 memdb" failed 4 times in a row with the same IRC problem.

"production leveldb" starts, but spews too fast to really understand what's going on.

Attempting redirection of standard output gives 'Exception in thread "main" java.io.IOException: unable to obtain console' and exits.

May I also have one suggestion: configure the program to default to "-nolisten", because it pops up the firewall configuration dialog first time it is being run. I let it have the firewall exception enabled, but I would really prefer if the program didn't ask for it first time.