Freicoin application developer best practices

[maaku]

I've written up on the Freicoin github wiki an article explaining how to properly handle demurrage in your own applications:

https://github.com/freicoin/freicoin/wiki/How-to-properly-handle-demurrage-in-applications

And a specific example of how to handle demurrage on an exchange:

https://github.com/freicoin/freicoin/wiki/How-to-implement-a-Freicoin-exchange

[Sunny King]

The exchange recommendation sounds overly complex and confusing to the users. Charging a percentage withdrawal fee based on difference between exchange wallet balance and exchange database balance is probably a lot simpler to implement and understandable to users.

So under this approach exchange records user balance same as the other cryptocoins, without any demurrange accounting. Withdrawal fee is a percentage value greater than 1 - (current wallet balance)/(sum of user balances). It can be automatically adjusted or fixed by the owner periodically.

[doublec]

So under this approach exchange records user balance same as the other cryptocoins, without any demurrange accounting. Withdrawal fee is a percentage value greater than 1 - (current wallet balance)/(sum of user balances). It can be automatically adjusted or fixed by the owner periodically.

Doesn't this have the issue that sell orders can exist without coins backing it? If a user lists 1,000 FRC for sale which is their exact balance on the exchange then on the next block they no longer have 1,000 coins to match the order.

[doublec]

Is there an issue with race conditions between a site and the blockchain for sites that hold balances like exchanges? Say a user of the exchange withdraws their balance of 1,000 FRC which the exchange has calculated as the amount at "present value". During the time of processing the withdraw the block height changes due to blocks being found. Now that 1,000 being sent is backed on the exchange by 999.998 (or 1,000 less demurrage). It'll go through because there are other user funds in the exchange wallet. But over time the exchange loses out.

If a withdrawal takes a few blocks to go through due to high transaction volume on the blockchain or malicious miners holding back exchange blocks will this results in a bigger difference between what the exchange expected and what actually happened?

The exchange could detect the transaction when it completes, find out the real value and adjust the users balance but then an exchange account can be in debt and the user can just leave and create a new account.

[crazy_rabbit]

Well at least it's not complicated. 

[maaku]

Now you've gotten into the mechanics of how demurrage works. Demurrage is calculated based on the reference-height, which is embedded in the transaction itself, not the height of the block containing the transaction. So the exchange would create the withdrawal transaction at the current block height, and that's the amount that would be go to the client and be reported in the client's transaction tab.

---

And I just realized that while discussed in depth on our own forums, this probably hasn't been made clear here: always check the reference-height of an incoming transaction to make sure it's close to what you'd expect. The creator of a transaction is allowed to select the reference height value.

Some people are probably sitting up in their chairs at reading that, thinking “oh my god that's exploitable!” Well it's a bit more complicated and some thought has been put into it. There are some new rules for validating transactions: the reference height must be greater than or equal to all of its inputs reference heights, and less than or equal to the block height, and coinbase transactions must have a reference height equal to the block height. And even if you purposefully constructed a transaction with an old reference height (there are no tools to do this currently), it'd still show up as less than the advertised amount in the recipient's balance.

However it is possible for someone to construct a new transaction with an old reference height, if they have access to suitably old outputs, such that it would show a larger amount on the transaction tab than is added to the recipient's available balance. More than anything else, that's why the transactions tab includes the Ref-height column - always check the reference-height of an incoming transaction.

We are considering client updates which would make it more difficult to pull off a demurrage fraud, such as visually calling attention to an unusually old reference height, for example. But the long-term solution is to add a payment/invoice protocol which specifies the request block height in a payment request. The received amount can then be checked against the requested reference height.

The payment/invoicing protocol is only just now being discussed on the bitcoin developer's mailing list, however, so it might be some time before that is implemented.

[Sunny King]

And I just realized that while discussed in depth on our own forums, this probably hasn't been made clear here: always check the reference-height of an incoming transaction to make sure it's close to what you'd expect. The creator of a transaction is allowed to select the reference height value.

Some people are probably sitting up in their chairs at reading that, thinking “oh my god that's exploitable!” Well it's a bit more complicated and some thought has been put into it. There are some new rules for validating transactions: the reference height must be greater than or equal to all of its inputs reference heights, and less than or equal to the block height, and coinbase transactions must have a reference height equal to the block height. And even if you purposefully constructed a transaction with an old reference height (there are no tools to do this currently), it'd still show up as less than the advertised amount in the recipient's balance.

However it is possible for someone to construct a new transaction with an old reference height, if they have access to suitably old outputs, such that it would show a larger amount on the transaction tab than is added to the recipient's available balance. More than anything else, that's why the transactions tab includes the Ref-height column - always check the reference-height of an incoming transaction.

Hmmm not sure I understand that, you didn't mention any restriction how close reference height must be to the block height, then it seems possible to avoid demurrage altogether by not combining multiple inputs and use the same reference height of the old coins. Not sure if this is what you mean by 'demurrage fraud.'

[maaku]

Hmmm not sure I understand that, you didn't mention any restriction how close reference height must be to the block height

There is none. If there were, then a transaction would be invalidated if it didn't make it in the blockchain in time. I think we'd agree that'd be a worse outcome.

then it seems possible to avoid demurrage altogether by not combining multiple inputs and use the same reference height of the old coins. Not sure if this is what you mean by 'demurrage fraud.'

Yes, but the fraud could only be pulled off if there's a human in the loop, and one which didn't bother to correlate. Any application (payment service, exchange, etc.) would properly account for demurrage and know that 100 coins sent from a reference height 1 year in the past is still only ~95 coins.

[Sunny King]

Yes, but the fraud could only be pulled off if there's a human in the loop, and one which didn't bother to correlate. Any application (payment service, exchange, etc.) would properly account for demurrage and know that 100 coins sent from a reference height 1 year in the past is still only ~95 coins.

But someone could release a client that split one send into multiple send transactions and eliminate demurrage? Wouldn't this cause the entire demurrage thing becoming a futile exercise? Did you say something about official client detecting this 'demurrage fraud' if the received coins having very old reference height?

[Sunny King]

So under this approach exchange records user balance same as the other cryptocoins, without any demurrange accounting. Withdrawal fee is a percentage value greater than 1 - (current wallet balance)/(sum of user balances). It can be automatically adjusted or fixed by the owner periodically.

Doesn't this have the issue that sell orders can exist without coins backing it? If a user lists 1,000 FRC for sale which is their exact balance on the exchange then on the next block they no longer have 1,000 coins to match the order.

The exchange wallet balance can be smaller than the sum of user balances. But after paying withdrawal fee the wallet balance should cover all user accounts.

However on second thought I think this scheme albeit simple would attract users to treat the exchange as 'safe-harbor' to reduce their demurrage. Then the withdrawal fee needs to go higher and higher causing damage to the popularity of the exchange.

The hard part seems to be how to avoid modifying an order on the order book.

[doublec]

Now you've gotten into the mechanics of how demurrage works. Demurrage is calculated based on the reference-height, which is embedded in the transaction itself, not the height of the block containing the transaction. So the exchange would create the withdrawal transaction at the current block height, and that's the amount that would be go to the client and be reported in the client's transaction tab.

There's still this race though, unless I'm misunderstanding things:

Say a user of the exchange withdraws their balance of 1,000 FRC which the exchange has calculated as the amount at "present value". During the time of processing the withdraw the block height changes due to blocks being found. Now that 1,000 being sent is backed on the exchange by 999.998 (or 1,000 less demurrage). It'll go through because there are other user funds in the exchange wallet. But over time the exchange loses out.

That is the exchange has calculated the balance of the user for the withdrawal based on current blockheight and the user clicks "Go". The freicoind rpc transaction is crafted just after the blockheight changes in the second between "Go" and the RPC call. Now the value of the coins withdrawn in the transaction is higher than the amount computed internally by the exchange. This is a small window of time, and small amounts, but in a long running exchange it'll happen.

I guess the exchange can avoid this by crafting the raw withdraw transaction itself, embedding the reference height it used for the demurrage calculatoin into it, and using the "sendraw" commands. That's a bit more work for the exchange however.

[maaku]

I guess the exchange can avoid this by crafting the raw withdraw transaction itself, embedding the reference height it used for the demurrage calculatoin into it, and using the "sendraw" commands. That's a bit more work for the exchange however.

That is what I had in mind. Demurrage forces us to be a little more exact in specifying behavior and dealing with these race conditions, but so far as I've seen it doesn't present unsolvable problems.

[jtimon]

I guess the exchange can avoid this by crafting the raw withdraw transaction itself, embedding the reference height it used for the demurrage calculatoin into it, and using the "sendraw" commands. That's a bit more work for the exchange however.

I guess the cleanest way to do this is to use the optional refHeigh argument on this issue https://github.com/freicoin/freicoin/issues/41
I was going to implement those since they were something easy to start with but I wanted to stop using Eclipse for coding at home first and I felt into the .emacs hole (also starting with conkeror, remapping my keyboard to an american-like one with more symbols I need and testing several tiling window managers). Sorry, I feel responsible for this feature being missing. I should had found the time. 

[jtimon]

I've created a pull request for those optional fields. I haven't tested it though as I allowed ubuntu to brake the build by updating and removing things.
It's little work, since the heavy part was to elevate nBestHeight substituting it with nRefHeight all around, but I hope it helps:

https://github.com/freicoin/freicoin/pull/83

I'll try to test it later, but maybe it was easier for some of you to test and make the necessary corrections if any.

[jtimon]

I've written this, in case someone's interested on issuing freicoin colored coins to enable a p2p atomic btc/frc market:

https://github.com/freicoin/freicoin/wiki/Freicoin-colored-coins