[Stratum] Overlay network protocol over Bitcoin

[kjj]

But I agree that some mechanism for negotiating required fees with miners would be interesting, too.

It may be interesting from the anthropological point-of-view: who's gonna fall for it. Technically the negotiation for inclusion in a block doesn't make sense: the blocks are won randomly. I guess from the game-theoretical point-of-view one could come up with an optimal strategy: clairvoyance.

I think the quote would be along the lines of: "We have averaged X blocks per day over the last week.  We expect our next block to happen within Y hours.  If you get your transaction to us in the next M minutes and it includes a fee of at least Z, we will include it in our next block (which should happen within 2*Y hours).  To ensure that your transaction reaches us, please submit it directly to our node at address A.B.C.D, port E."

The quote doesn't promise that the miner is going to get the next block, they are only promising that if they do get the next block, your transaction will be in it, provided that the fee meets their criteria.

In a busy world, I would expect that the biggest miners (the ones that average like one block per hour or more) will start ignoring low fee transactions, while the smaller miners would accept a lot more.  A spender could check with a bunch of miners and make the time/fee trade off that they want.

Deepbit wants 0.01 BTC to include it in the next 30 minutes, but Joe's Mining Pool and Bait Shop will do it for 0.001 BTC in the next day.  What is more important to you, time or money?

[1713557241]

1713557241

[1713557241]

1713557241

[1713557241]

1713557241

[2112]

A spender could check with a bunch of miners and make the time/fee trade off that they want.

I'll pretty much believe that in the Bitcoin milieu anything is possible: if somebody builds a "pig in a poke" ("Katze im Sack") market it will find willing participants. Those phrases became proverbial because there was a continuous demand for it throughout the history. There will be natural extensions for them implemented using Bitcoin. That's what makes Bitcoin such interesting from the anthropological perspective. 

Deepbit wants 0.01 BTC to include it in the next 30 minutes, but Joe's Mining Pool and Bait Shop will do it for 0.001 BTC in the next day.

What a charming old-fashioned prediction! Deepbit will merge with an exchange and it will offer negative effective transaction fees to the users of the exchange.

Joe will offer "Mining Pool and Guaranteed-payback Three-card Monte": that's the market demand amongst the Bitcoin miners.

[MoonShadow]

Deepbit wants 0.01 BTC to include it in the next 30 minutes, but Joe's Mining Pool and Bait Shop will do it for 0.001 BTC in the next day.

What a charming old-fashioned prediction! Deepbit will merge with an exchange and it will offer negative effective transaction fees to the users of the exchange.

Joe will offer "Mining Pool and Guaranteed-payback Three-card Monte": that's the market demand amongst the Bitcoin miners.

This is an utterly rediculous statement.

[casascius]

I assume that part of an overlay network would be to provide a means for nodes to tell one another about one another's existence (e.g. I'm a bitcoin node accepting incoming connections at 1.1.1.1:8333).  Also it would be to exchange signed messages of useful value (e.g. signed message from deepbit: I have seen block 123456, signed message from MtGox: rates as of 12/12/12 11:11:11: buy 11.11 sell 11.11 etc).

If that infrastructure were in place, it wouldn't be much of a stretch for a signed message from deepbit to include some sort of message broadcasting its fee schedule, or broadcasting the url of an rpc that can give a fee quote for any given proposed transaction.

[slush]

I assume that part of an overlay network would be to provide a means for nodes to tell one another about one another's existence (e.g. I'm a bitcoin node accepting incoming connections at 1.1.1.1:8333).  Also it would be to exchange signed messages of useful value (e.g. signed message from deepbit: I have seen block 123456, signed message from MtGox: rates as of 12/12/12 11:11:11: buy 11.11 sell 11.11 etc).

Yes, signing and relaying (proxying) signed messages is already possible, exactly for transmitting messages like exchange rates.

[HostFat]

Are you still working on a BitcoinSpinner version compatible with Electrum servers?

[slush]

I'm not working personally on that, but friend of mine is. However I don't know actual status of his project...

[slush]

Quick performance test of Stratum server implementation on my Asus EEE netbook:

TCP transport: cca 450 rq/s
HTTP transport: cca 130 rq/s

Every request is one roundtrip, including serialization and deserialization of request/response in testing client, so server part itself is at least 2x faster. However the overhead of HTTP layer is significant, as I expected. I compared those numbers with bitcoind's RPC (getblockcount, which is basically blank RPC request): ~100 rq/s.

Asus EEE is lowend machine and this test is calling RPC method in serial, so the test is far from real-world use, but I'm expecting at least 2x higher performance of this test per CPU core on standard server machine.

[jetmine]

client -> (some transport) -> overlay server -> HTTP (firstbits.com don't support https yet!) -> firstbits.com

[...]

I has been thinking about using GPG mechanism, which is widely used platform even for distributing trust, but it looks prettyheavy for such purpose. Not sure if it's possible to check GPG signatures in $3 AVR processor.

[...]

However I see real issue in needed CPU time for signing. On my laptop, one signature takes around 200ms. It's low-end one, but it will take some time even on high-end servers. Any idea how to solve it?

The obvious answer to this question is:

a) Use an algorithm which allows signature verification with low CPU requirements.  RSA with E=3 is one such algorithm.  It is possible to verify a signature on a low-end CPU like AVR without problems.

b) On the server side, scale up as required to meet the desired signature creation throughput.  See the next item for how this is not necessary (except maybe when you plan to run your server on AVR too).

c) Include into the signature only fields that are essential.  Firstbits is assumed to be unique and will not change throughout the lifetime of bitcoin.  Therefore the essential fields are 1) Firstbits address, 2) Full address, 3) Who signed this (you), 4) Signature version/date/generation method.  What else is necessary?  I believe nothing else (but I have never used firstbits so I cant be sure).  With this, you can generate the signatures as requests trickle in, and cache them on your server.  When an address is requested multiple times, you can satisfy from the cache (without further CPU effort).  Soon you will have all (relevant) addresses in your cache and will not have to sign any new addresses, at least until the the blockchain grows (which happens at a predictable rate).

Good luck

[slush]

jetmine, thanks for inputs. Quick answer:

a) Use an algorithm which allows signature verification with low CPU requirements.  RSA with E=3 is one such algorithm.  It is possible to verify a signature on a low-end CPU like AVR without problems.

Signing will be probably necessary only for some type of messages so the speed of verifying on the client won't be crucial. I decided to implement ECDSA signatures because ECDSA library will be presented on the client anyway for signing transactions, which eliminate yet another dependency.

c) Include into the signature only fields that are essential.  Firstbits is assumed to be unique and will not change throughout the lifetime of bitcoin.

Actually I did the oposite, I added timestamp into the signed data, to protect system against replay attacks. However now I'm rethinking the concept, as the same request should lead into the same result, at any time. Those APIs where replay attack is potentially dangerous should add timestamp as the part of the request (signature is created from both request and response).

So - thanks for suggestion!

and cache them on your server

This can be doable with some type of messages even with timestamp in the current implementation, for messages like market quotes.

[slush]

I updated Google document witch chapter describing HTTP Poll / Push transports. Any suggestions?

I'm also thinking about HTTP Long-polling transport (it's actually blocking extension of HTTP Poll transport, which enable realtime communication from server to client over HTTP without any need of polling or callback URL), but I want to provide long polling over the same connection, so I need to test HTTP pipelining firstly. In the theory HTTP pipelining should be working already in the Stratum implementation, but not so much client libraries support that...

ripper234, you were interested in implementing Stratum client library in Java with HTTP transport. Is current specification detailed enough for finish the job from your side? Everything is already implemented and commited into the Git so it should be very easy to install and run locally, but I'll provide Stratum instance today or tomorrow for more comfortable development.

[ripper234]

I updated Google document witch chapter describing HTTP Poll / Push transports. Any suggestions?

I'm also thinking about HTTP Long-polling transport (it's actually blocking extension of HTTP Poll transport, which enable realtime communication from server to client over HTTP without any need of polling or callback URL), but I want to provide long polling over the same connection, so I need to test HTTP pipelining firstly. In the theory HTTP pipelining should be working already in the Stratum implementation, but not so much client libraries support that...

ripper234, you were interested in implementing Stratum client library in Java with HTTP transport. Is current specification detailed enough for finish the job from your side? Everything is already implemented and commited into the Git so it should be very easy to install and run locally, but I'll provide Stratum instance today or tomorrow for more comfortable development.

I might have some time to look at this next weekend, but probably not before

[slush]

I'm proud to announce first instance of Stratum server on stratum.bitcoin.cz, ports 3333 (TCP transport) and 8001 (HTTP transport). Although there's not a documentation about exposed services, some samples in client.py and client2.py together with source codes in /server_repository/ directory may be useful for hardcore hackers.

In the near future I'll configure Nginx to handle HTTP transport on port 80 and add SSL certificate to enable HTTP transport over SSL on port 443, but this should be enough as a playground for some interested parties.

Almost everything on the protocol level is implemented and I'm interested in any feedback, especially in using the HTTP Poll and Push transports.

[Red Emerald]

How much overlap is there between this and electrum?  I noticed in server.tac that the application is called "electrum-server" but it looks like the code is being written separately.  BTW, Your code looks really clean.

I'll play around with getting some javascript to connect to the server when I have time.  Right now I'm working on a cgminer monitor.

[jetmine]

Those APIs where replay attack is potentially dangerous should add timestamp as the part of the request (signature is created from both request and response).

PS to the earlier said: Make sure you always include a signature generation or date into the signed portion, even when the API doesnt immediately require it.  In the firstbits example, imagine you would find a bug in the code someday and a few of the addresses were matched in a wrong way.  With date or generation information you could then "revoke" the old body of signed snippets by requiring date/generation > X both on the server (to trigger recreation of data+sig) as well as on the updated clients (as security measure).

As for market stats and other things you mentioned, of course you could also cache those pieces of information.  Just make up a time grid, in accordance to how often you poll the information yourself.  It doesnt make sense to sign it at a higher rate than they can change (which is almost entirely dependant on your pollrate).

You can then hand out the same signed piece(s) many times to many clients.

You can also hand out old (cached) pieces, to give historical information to the clients.  However, it makes sense to "compress" the many pieces of information within one timeperiod into a single (newly signed) packet once the timeperiod is over.  This reduces the amount of signature verifications on the client side, yet doesnt greatly increase the signature load on the server.

For example, after 15 minutes, you combine the 15 1-minute pieces into one 15-minute piece.  Every 4 hours you do the same.  Every day you combine the whole day, every week, every month and every year.  With every higher timeperiod you can also reduce the "resolution" to avoid excessively growing the size of the datasets.  The client can then easily assemble price information for practically every imaginable time ranges, with only few signature verifications.  All just from cached information and nothing individually signed ("just for them").

For moments where you cant query prices yourself (e.g. network problems), you could insert a packet saying so, and sign that.   The client can then check its local time and know that it has complete information from your server, despite the lack of data.

But make sure that you dont put too much effort on features that maybe arent useful later on.  If you poll prices from doubtful sources using insecure protocols and then sign it, the high quality signature from your server wont serve much purpose.  It may even be misleading then..

Always follow the KISS principle (Keep It Simple, Stupid).  In electronics we say that the best component is one that you can eliminate.  Once eliminated, the component will not have solder problems, it will not fail throughout the whole lifetime of the product, and it adds 0.00 to the BOM.  The same goes for software.  If you create an elegant design on the API level, your backend can be beautifully simple, and your clients can be too.  But its all up to knowing (in advance) where you want to go and what you want your software to do.

This is the hard part of it, and choosing pieces like signature algorithms just for "familiarity" can be a lottery in this respect.  Maybe such a tiny decision can bring your network down when it is supposed to just scale up nicely.  I dont know the details of what you want to do (and maybe you dont fully know either).  So I am sorry I cant give detailed advice.

Good luck with the project.

[BitcoinBug]

I love this project. Following...

[slush]

How much overlap is there between this and electrum?  I noticed in server.tac that the application is called "electrum-server" but it looks like the code is being written separately.

Yes, the codebase is completely different. Originally I started this project as a part of Electrum client, but then realize that it has some bigger potential than be just a backend for one particular client, so I rename it to Stratum not confuse people. Currently it shares the motivation with Electrum server and I'm also going to use same compontents for the initial implementation like Abe for blockchain indexing, but other things are different. Btw thanks for pointing me to "electrum-server", I fixed the typo :-).

BTW, Your code looks really clean.

Thanks, I'm simply not smart enough to write complicate code .

[slush]

PS to the earlier said: Make sure you always include a signature generation or date into the signed portion, even when the API doesnt immediately require it.  In the firstbits example, imagine you would find a bug in the code someday and a few of the addresses were matched in a wrong way.  With date or generation information you could then "revoke" the old body of signed snippets by requiring date/generation > X both on the server (to trigger recreation of data+sig) as well as on the updated clients (as security measure).

Thanks for suggestion, I think it's good compromise - add timestamp of signing into the message, but use it just as an additional information and don't force clients to throw away messages with old timestamps. In this case client can decide if he cares about timestamp of signature generation or not.

For example, after 15 minutes, you combine the 15 1-minute pieces into one 15-minute piece.  Every 4 hours you do the same.  Every day you combine the whole day, every week, every month and every year.  With every higher timeperiod you can also reduce the "resolution" to avoid excessively growing the size of the datasets.  The client can then easily assemble price information for practically every imaginable time ranges, with only few signature verifications.  All just from cached information and nothing individually signed ("just for them").

Well, I feel this isn't the responsibility of the protocol itself, it's more about design of exposed service. As an example, one service can provide simple method for retrieving actual information and second method accepting some range specification as an input for providing historical data. Of course that response of such method with composite result will be signed only once. I think that don't include some extended algorithms to the protocol for solving edge cases is going exactly in the KISS way.

Edit: Timestamp is the part of signed message again, in enlightened form than original implementation.

[slush]

I just implemented support for specifying vendor in the RPC method name, in the form "some.service[vendor].method".

Longer answer, especially for potential service developers:
Stratum is designed in the way that one service may be provided by many independent vendors. As an example, every exchange provides only market quote which is actively traded there, obviously. In this case, Stratum server operator or even exchange itself can implement the service for broadcasting their quotes and let the client choose the vendor from which he wants to receive quotes.

1. There's already discovery service, which provides listing of available vendors for some specified service: discovery.list_vendors('exchange.quotes') may return ['mtgox.com', 'tradehill.com']

2. Listing of two vendors means that there are two implementations of ExchangeQuotesService (located in server_repository in current server implementation), which provides the same service API for both independent sites (mtgox provides websocket, tradehill only polling?). Those two classes may contain completely independent implementation, but acts as the same service for Stratum client.

3. Client wants to receive quotes from mtgox, so he call exchange.quotes[mtgox.com].subscribe('usd'), which tell Stratum server to start broadcasting USD quotes from mtgox service implementation. If the client wants to receive tradehill quotes instead, he call exchange.quotes[tradehill.com].subscribe('usd') instead, but he don't need to care that internal implementation is different.

Edit: When brackets with requested vendor aren't specified, Stratum will use default service for given service type (default service class is marked as default=True). In the example above, it could be bitcoincharts.com as general provider of all quotes (although using specific vendor binding have some benefits, bitcoincharts is just another layer which can be down or overloaded, like during yesterday peaks on mtgox).

[slush]

I finally found some time to learn basics of IRC protocol. Thanks to it, I implemented IRC peer lookup mechanism into Stratum server. See node.get_peers() method...