Assurance contracts

[titeuf_87]

Hello everyone,

After seeing Mike Hearn's talk at the conference I wanted to play a bit with some of that stuff so I started working on an assurance contracts proof of concept.

Using bitcoin 0.7 and the raw transaction API I got this working on testnet-in-a-box. I haven't tried this on the main bitcoin network.

This is a small python script that does three things:

1) Creating a new assurance contract.

Code:

$ python pledge.py createcontract miA6bqrz9thieUpd5rZmHe1XpTmY6DLjot 7500000000
Contract tx:
Amount:  75
To:  miA6bqrz9thieUpd5rZmHe1XpTmY6DLjot
01000000000100eb08bf010000001976a9141cf69384eb6b27a8161b59577f41310a93b9c14388ac00000000

2) Adding pledges (this should be done by the various people wanting to pledge some bitcoins):

Code:

$ python pledge.py pledge 01000000000100eb08bf010000001976a9141cf69384eb6b27a8161b59577f41310a93b9c14388ac00000000 5000000000
Sent 50BTC to address n21Nw4ikZrURVxaydZwYrPmdDyd8J8ZdQw
(private key: cVNe9DSbZPe71pzwWpRbiR1n4NQ2Bdqi67g7HGiJCzARtX66nefY)
TXId: 0ae14dc7c016d8bce090cfaa2eb88ce1e2818f24d3372c5682118ff5a68e8200

TXOut: 0ae14dc7c016d8bce090cfaa2eb88ce1e2818f24d3372c5682118ff5a68e8200:0
Signing transaction...

Signed input:
00828ea6f58f1182562c37d3248f81e2e18cb82eaacf90e0bcd...

3) The person that made that contract needs to collect all inputs from everyone and put them in a single text file.
If there are enough pledges they can be combined and the final transaction sent to the bitcoin network.

Code:

$ python pledge.py finalize 01000000000100eb08bf010000001976a9141cf69384eb6b27a8161b59577f41310a93b9c14388ac00000000 inputs 
Sending raw tx...
010000000200828ea6f58f11...

This is still very much a proof of concept: command line application, having to share inputs, big strings that needs to be copied around, ... But it works

The code is on https://github.com/titeuf87/btcassurance
Parts of it are copied from bitcoin-python, pynode and pywallet.

I think the next thing for this is to divide it in a server/client model: a server that holds assurance contracts and a client that allows people to pledge bitcoins to one of those contract and send a signed input over.

[Mike Hearn]

That's great! Here are a few thoughts.

I strongly suggest to anyone who wants to implement these sorts of apps to consider the language and frameworks you're using beforehand. Python is great for prototyping and scripting. It is a poor choice for building user-friendly desktop apps that perform well. C++ apps are the best for that, but also harder to write. Java sits somewhere in the middle.

Anything that depends on bitcoins RPC mechanism means you need a full node running, and that will become less and less common in future, especially for end users.

The hard part of implementing these systems isn't really the crypto, it's all the stuff you need to layer on top to build a slick and straightforward user experience. Obviously starting with the command line is easy but you should really think ahead to how you can let your parents use it.

If I was implementing this system, I'd write start with a GUI Java app using bitcoinj. Not just because I wrote it and so am familiar with it, but because it's able to directly connect to the P2P network and sync a wallet to the chain very fast compared to a full node. Also because the APIs should make it easy to set up the transactions you need. Having a separate wallet is useful because it means you can easily move money in and out, and pledge the same money to the same contracts (a useful ability when there are multiple providers competing to provide a service and you don't know which one will achieve the target first). Then I'd come up with a set of protocol buffers that described an assurance contract, how to submit it to the server and so on. I'd use protocol buffers for even very simple protocols because that way you get easy extensibility and the protocol can be easily described in, eg, a standards document.

bitcoinj would need a couple of minor extensions to craft the right kind of transactions, but if somebody was serious about making a real product out of this that was going to be slick and easy to use, I'd happily implement those extensions and write some example code.

[titeuf_87]

I strongly suggest to anyone who wants to implement these sorts of apps to consider the language and frameworks you're using beforehand. Python is great for prototyping and scripting. It is a poor choice for building user-friendly desktop apps that perform well. C++ apps are the best for that, but also harder to write. Java sits somewhere in the middle.

I don't want to start a language debate here as there are a lot of arguments made already both in favor and against Python. As an Ubuntu user though, a lot of applications I use are made in Python and they work just fine.
The biggest downside is probably portability of a GUI-framework: GTK is horrible in Windows and no idea if it even works on Mac OS X. Qt is probably a better option here, but I don't have any experience with this.
C++ would be the best option here, combined with Qt. But I'm only good with Python and .Net.

Anything that depends on bitcoins RPC mechanism means you need a full node running, and that will become less and less common in future, especially for end users.

True. Ideally you would just have a website where people can send bitcoins to a certain bitcoin address to pledge how many bitcoins they want and it would not work any differently than sending bitcoins to a normal bitcoin address.
But I doubt something like that is possible.

The hard part of implementing these systems isn't really the crypto, it's all the stuff you need to layer on top to build a slick and straightforward user experience. Obviously starting with the command line is easy but you should really think ahead to how you can let your parents use it.

I already noticed that Making this proof of concept was easy, but I've been thinking about how to progress and while there are different ways, they aren't as easy.

If I was implementing this system, I'd write start with a GUI Java app using bitcoinj. Not just because I wrote it and so am familiar with it, but because it's able to directly connect to the P2P network and sync a wallet to the chain very fast compared to a full node. Also because the APIs should make it easy to set up the transactions you need. Having a separate wallet is useful because it means you can easily move money in and out, and pledge the same money to the same contracts (a useful ability when there are multiple providers competing to provide a service and you don't know which one will achieve the target first). Then I'd come up with a set of protocol buffers that described an assurance contract, how to submit it to the server and so on. I'd use protocol buffers for even very simple protocols because that way you get easy extensibility and the protocol can be easily described in, eg, a standards document.

I was thinking about something similar, except without any blockchain at all on the client side. Since there needs to be a central server anyways to collect all signed inputs, that central server can keep track of the blockchain too.

I imagine something like that: a webserver like kickstarter where people can see all the projects they can pledge bitcoins to. If someone wishes to pledge to a project they can use a client application. This client application is very basic:
1) It creates a new bitcoin address and asks the user to send bitcoins to it.
2) It notifies the server about this address: when bitcoins are sent to it, the server will let the client know and the client will sign a pledge for the assurance contract and send it back to the server.
3) Whenever the server got enough pledges it'll broadcast the final transaction to the server
4) If there aren't enough pledges and the user wants their bitcoins back then the client can create a new bitcoin transaction and send it to the server who will broadcast it.

This is, I think, still safe since the private key to this new bitcoin address is kept in the client application and the server never gets access to it.

Using bitcoinj to directly connect to the bitcoin network would make some of those steps unnecessary, at the cost of the client having to handle the blockchain. I'm not really sure that extra cost is worth it, even if it goes a lot faster than a full node.

Also, with protocol buffers, you mean to use that for the communication between the client and the server, right? If so, why not just use a http server and json? I don't know much about protocol buffers except from looking at the examples.

Thanks for your thoughts

[Severian]

Donation sent. Assurance contracts are going to be one of the foundations of Bitcoin's acceptance by the larger world.

Speaking as a non-developing luser, please continue with this line of work.

[kjj]

Cool!

Are you following BIP10?

[Mike Hearn]

As an Ubuntu user though, a lot of applications I use are made in Python and they work just fine.

Yes. Python works very well on Linux because it is shipped out of the box, has good bindings to the native toolkits, etc. But most people don't use Linux.

I was thinking about something similar, except without any blockchain at all on the client side. Since there needs to be a central server anyways to collect all signed inputs, that central server can keep track of the blockchain too.

Yes, it could do that. It complicates the client/server protocol though, and in future I'd hope to see assurance contract support integrated into wallet software. As it's already doing the block chain tracking, it'd be easier for those clients to just submit the transaction directly.

You can set up a basic GUI wallet app in bitcoinj with only a few lines of code, it's not very complicated.

2) It notifies the server about this address: when bitcoins are sent to it, the server will let the client know and the client will sign a pledge for the assurance contract and send it back to the server.

The server has to indeed track the block chain no matter what, so it could pass transactions back to the client for signing. But there's already a protocol that lets a client find transactions given some keys/addresses - the regular Bitcoin P2P protocol. It's weird to reimplement that. Bear in mind, if you don't care about handling re-orgs, the default P2P protocol is quite easy to use. But you probably do care about re-orgs, otherwise you'll end up with weird edge case bugs.

This is why I recommend just using a pre-written Bitcoin library. Switching out the bitcoin P2P network for something HTTP+JSON based doesn't change the fundamental rules of the system, it doesn't let you ignore re-orgs and other details of the chain consequence-free.

3) Whenever the server got enough pledges it'll broadcast the final transaction to the server
4) If there aren't enough pledges and the user wants their bitcoins back then the client can create a new bitcoin transaction and send it to the server who will broadcast it.

Why should the client depend on the server to help them undo their pledge? That seems like an inherent conflict of interest. Again, just talking to the P2P network when possible solves this issue.

Also, with protocol buffers, you mean to use that for the communication between the client and the server, right? If so, why not just use a http server and json?

You can use HTTP if you like. Why protobufs? Because I think they're better than JSON. Firstly there's a simple schema language that's easy to read. Most of the time when I see JSON examples they roll their own way to describe the protocol and rely heavily on examples. Secondly there are protobuf compilers that generates wrapper code for you based on the schema language, for almost any programming language you can name. So you can work with data structures directly instead of awkward JSON data types. Thirdly it's efficient and robust, especially when transmitting binary data (which is common in Bitcoin protocols), there are no weird encoding quirks that can bite you like with JSON.

In short I can't see any reason to use JSON for anything unless you're communicating with a web browser and are OK with just eval()ing the response. But even then sometimes it's unsafe.

Also protobufs are gaining pretty good adoption, there are lots of libraries and things that integrate with them. Eg, the Netty async IO library in Java can handle protobuf+HTTP natively so there's almost no code needed.

I can guess the protocol might look like

Code:

// Stored in a file and downloaded from some web server/email attachment/etc.
message ContractDescription {
  required bytes output_script = 1;
  required uint64 target_value = 2;
  // An HTTP or HTTPS location where a Pledge message can be POSTd.
  // Can be missing in protocol variants where this is provided by context.
  optional string pledge_submit_url = 3;

  // A human-readable description of what the contract is about.
  optional string description = 4;
  // A 600x300 PNG that can be displayed in the GUI alongside the description.
  optional bytes logo_image = 5;
}

message Pledge {
  // Sends target_value funds to output_script
  required bytes serialized_tx = 1;

  // A human-readable message (maybe of encouragement) to the entrepreneur
  optional string message = 2;
}

[titeuf_87]

Donation sent. Assurance contracts are going to be one of the foundations of Bitcoin's acceptance by the larger world.

Speaking as a non-developing luser, please continue with this line of work.

Thanks, it's appreciated

Cool!

Are you following BIP10?

I didn't know about this BIP. I looked at it and at first glance it seems good: there is a transaction that needs to be signed. But I don't think it will work for this use case: every user wants to add transaction inputs to this, not just sign them. And they need to be signed using "anyonecanpay".

long post...

Thanks You convinced me to use the bitcoin p2p network itself. I do like the idea of an insurance contract being just a file you can download (which is also what Kjj suggested in his post). Your protobuffer example made this really clear how it can work. This makes the client/pledge application able to work more independently and not tied to one particular server.

I've played around a bit with bitcoinj earlier today (while also brushing up my Java knowledge, it's been a while). From a quick look at the source, it seems that creating the contract transaction would already be possible, but the signing of inputs using anyonecanpay isn't possible yet?

[kjj]

Cool!

Are you following BIP10?

I didn't know about this BIP. I looked at it and at first glance it seems good: there is a transaction that needs to be signed. But I don't think it will work for this use case: every user wants to add transaction inputs to this, not just sign them. And they need to be signed using "anyonecanpay".

I would talk to etotheipi about this, and try to use essentially the same format.  The details are slightly different, but the idea is very similar: you are collecting signatures for later combination.

[Mike Hearn]

I've played around a bit with bitcoinj earlier today (while also brushing up my Java knowledge, it's been a while). From a quick look at the source, it seems that creating the contract transaction would already be possible, but the signing of inputs using anyonecanpay isn't possible yet?

If you look at Transaction.signInputs() there is a boolean hardcoded to false called anyoneCanPay. There's also an assert at the top checking you use SIGHASH_ALL, as I never tested the other flags.

I think it'd be some fairly simple modifications to add support. Basically you would extend Wallet.SendRequest to have a new parameter like "inputValue". By default it'd be unset/set to a dummy value like -1 and then obviously the input value would be >= output value as now. If explicitly set then wallet.completeTx() would add enough inputs to reach the target value and then call tx.signInputs(SigHash.ALL, this, true); where the last boolean param would be one you add that'd set anyoneCanPay.

One thing that needs careful thought is what happens if you can't make the input target value exactly. Adding change outputs won't work properly, so you'd need to first construct a transaction that gives you an output of exactly the right value, and then another transaction to pledge it. That would complicate the API somewhat if done automatically, so I'd suggest just throwing an exception in that case and expecting the app to handle it at a higher level.

If you want to discuss it further let's take the design talk to the bitcoinj mailing list.

[titeuf_87]

I haven't been able to work much on this: only a few hours during the weekends.

Bitcoinj seems to work perfectly for the client (with some changes) but I've been thinking about the server now: I don't think either bitcoinj or the satoshi client have an easy way to know if a certain transaction output is spent?

This is needed so that when there are enough pledges the final transaction can be made with only valid, not already spent, inputs and broadcasted to the bitcoin network.

[Mike Hearn]

If a transaction is in the wallet and "pending", then it will be checked whenever a new block is found to see if any transactions in the block double spend an input of any transactions that are pending. I think there are still a few holes in this (eg multiple pending transactions that depend on each other), but I think if you add the pledges to the pending pool in the wallet then you can listen on the WalletEventListener.onTransactionConfidenceChanged() event, or the TransactionConfidence.Listener events, to see if a tx goes DEAD. If it's DEAD then it means it got double-spent into oblivion.

In the case of assurance contracts this is not necessarily a problem, the user may have pledged the same money to multiple projects on a first-come-first-serve basis.

[titeuf_87]

Thanks, you're helpful as always Mike

This seemed to work, I never thought of adding the pledges to the wallet of the person making the assurance contract since they aren't actually valid transactions (input < output).

[Mike Hearn]

Right, obviously just putting pledges into a wallet is insufficient. The pending transactions will be checked for double spends and you can learn about them via event listeners, but you still don't know what the input values are so you can't actually construct a spend with such a wallet.

I think we'd need the following changes:

The TransactionInput class would need to support caching of the values of the connected outputs. When a transaction is extracted from the chain normally the input values don't matter, you only care about the output values. For pledges you do care. Protobuf serialization would need to be upgraded so you can record that as well.

We still have the problem as you stated of knowing what the output is. One is that bitcoind could get a new JSON-RPC to inspect the set of unspent outputs. It means some C++ hacking. Other projects can benefit too. Alternatively, Matt Corrallo has upgraded bitcoinj to be fully validating. I am reviewing and merging his code at the moment. Once that work is merged, it could all be done within Java. I think that approach is probably superior as you have enough control to avoid some tricky race conditions.

The Wallet class needs to understand what to do with pledges. I guess a wallet could contain both regular valid transactions and pledges. When you try and construct a spend, it would gather all the pledges that shared an identical output and see if the assurance contract can be completed. If so then it's counted towards your balance and will be used to construct a spend. If not, then no. I think TransactionConfidence might need an extension to reflect that the pledge tx is completely invalid.

One question is, what if you have a wallet with regular funds and some pledges, and you construct a spend that wants the value of all of them? You can merge the pledges together, but you can't then merge that completed contract together with another arbitrary transaction. I think the Wallet API needs to be upgraded to support creating multiple transactions as part of a spend. It doesn't play so well with the current design Alternatively you could just impose some basic restrictions for now, like stating a wallet can contain pledges XOR regular funds. It's probably not an issue in practice.

[titeuf_87]

Earlier I was just thinking about detecting double spends and I didn't think further about actually knowing how much has already been pledged.

It is possible to fulfill the assurance contract even if you don't know how much each input is worth but that's not a road I want to take: it involves bringing in all pledges together in one transaction, submitting it to a bitcoin node and if it somehow gets in a block then we know that we had enough pledges. But this has so many downsides that I won't try this.

The TransactionInput class would need to support caching of the values of the connected outputs. When a transaction is extracted from the chain normally the input values don't matter, you only care about the output values. For pledges you do care. Protobuf serialization would need to be upgraded so you can record that as well.

Agree, this will be needed here.

We still have the problem as you stated of knowing what the output is. One is that bitcoind could get a new JSON-RPC to inspect the set of unspent outputs. It means some C++ hacking. Other projects can benefit too. Alternatively, Matt Corrallo has upgraded bitcoinj to be fully validating. I am reviewing and merging his code at the moment. Once that work is merged, it could all be done within Java. I think that approach is probably superior as you have enough control to avoid some tricky race conditions.

I'll wait till this is implemented in bitcoinj, especially if the code's already written

The Wallet class needs to understand what to do with pledges. I guess a wallet could contain both regular valid transactions and pledges. When you try and construct a spend, it would gather all the pledges that shared an identical output and see if the assurance contract can be completed. If so then it's counted towards your balance and will be used to construct a spend. If not, then no. I think TransactionConfidence might need an extension to reflect that the pledge tx is completely invalid.

There also needs to be a distinction to know if the assurance contract sends the bitcoins to an address owned by that wallet or not.

The way I imagined this is that people can create assurance contracts on a website for a certain amount of bitcoins to be sent to a certain bitcoin address that they choose. The server manages the collecting of pledges and creates the actual transaction when there are enough. At no point will it be able to spend any of the bitcoins, which I think is a big plus as if the server gets cracked into then no one loses any bitcoins.

One question is, what if you have a wallet with regular funds and some pledges, and you construct a spend that wants the value of all of them? You can merge the pledges together, but you can't then merge that completed contract together with another arbitrary transaction. I think the Wallet API needs to be upgraded to support creating multiple transactions as part of a spend. It doesn't play so well with the current design Sad Alternatively you could just impose some basic restrictions for now, like stating a wallet can contain pledges XOR regular funds. It's probably not an issue in practice.

Creating more than one transaction as part of a spend would also be nice too for when an user does not have the correct output values in order to pledge (for instance user has a 20 and 30 BTC transaction but wants to pledge 25 BTC)
But I won't worry about it for now as this is not really needed, just a nice bonus

[ffcitatos]

I'm sorry to invade the discussion with something not related to the above-mentioned implementation, but nevertheless: don't you consider the fact that assurance contracts are easily exploitable to make their use quite limited?

What I have in mind is that (in the language of https://en.bitcoin.it/wiki/Contracts example 3) after starting a fundraiser for 1000 BTC, and noticing that only 100 BTC will be raised by the deadline, the (fraudster) entrepreneur can sign a pledge for 900 BTC her/himself, and therefore get the 100 BTC as opposed to these coins returning to the donors.

Any comments are appreciated.

[Mike Hearn]

You can combine assurance contracts with mediation if you don't trust the entrepreneur to honestly fulfill the terms (the output is a threshold signature).