[UPDATE: 2015-05-10] Bitcoin Core soft-fork "No Forced TX Fee" v0.10.1 available

[Matt Corallo]

I wrote a proper Fee verbage and handling overhaul a while ago which allows just this (in the options page).
https://github.com/TheBlueMatt/bitcoin/tree/feefix and https://github.com/bitcoin/bitcoin/pull/289 and http://forum.bitcoin.org/index.php?topic=10923.0
sgimenez updated it to Wallet Class at https://github.com/TheBlueMatt/bitcoin/pull/2.
This doesnt have any kind of crazy side effects, and ONLY changes UI, nothing about relaying of backend stuff.

That said, as the patch makes clear, "You may override this option, however it is likely that your transactions will not be relayed through the network and will never be confirmed." so use any such fork at your own risk.

[1714174872]

1714174872

[1714174872]

1714174872

[1714174872]

1714174872

[1714174872]

1714174872

[1714174872]

1714174872

[1714174872]

1714174872

[Paperweight]

So, developers, what's the solution if...

1. You run an exchange where your clients have accounts in your wallet.
2. A malicious client / competing exchange operator makes many small deposits to his account.
3. The malicious client withdrawals all of his money at once and the huge fee gives his account a negative balance, which eats into your reserves.
4. He opens a new account and repeats the process until you're bankrupt and owe your other users their bitcoins back, which you now don't have.

Will any of these solutions work?
- Patch to pay no fees and only allow withdrawals of bitcoins with > X confirmations.
- Institute some convoluted system that disregards small deposits.
- Patch to estimatetxfee and subtract that from withdrawals before sending them.

Or what?

[gmaxwell]

Until one or both of those are done it is EXTREMELY DANGEROUS to remove the fee requirements. I hope you're prepared to recover all the stuck coins your patch will create.

To this day i was using 0.3.20 all the time and guess what: no coins were lost. The patch i created reverts the fee forcing behavior to 0.3.20 and that is all it does.
Also, on http://bitcoincharts.com/bitcoin/ there is only one transaction that is due 20 days, but it contains coins from unconfirmed transactions. Really bad.
So i guess this isn't so extremely dangerous after all, you just need to be a little more careful and only send stuff after you get 7 confirmations or more.

No— Bitcoincharts won't show any transaction which looks too much like DDOS. For example, it won't show a fee-less transaction with a 1e-8 output.  This is because the node feeding that page imposes the same anti-DOS rules as other nodes and it drops those transactions on the floor the moment they come in. Also because a sufficiently "bad" transactions will seldom make it to that host in the first place.

I've verified this myself.  In fact it wasn't even showing transactions that were perfectly fine under .23 rules, like transactions with <0.01 btc outputs and a 0.0005 fee, because it hadn't been upgraded yet.

Moreover, the reason it contains coins from unconfirmed transactions but not the older unconfirmed transaction itself is probably because the unconfirmed transaction is stuck.  It's perfectly valid to spend unconfirmed coin, you'll just need to be mined at the same time or later.   But you can see how producing transactions that won't ever confirm isn't just bad for you, it's bad for other people because these transactions are like a cancer that spreads when they show up as the only coins in someone's wallet and they get respent.

The likely reason that you haven't had any issues is simply because the cases where fees are forced are fairly rare, especially after you've been running bitcoin with a non-zero balance for a few weeks.  You've likely just not had many cases where a fee would have been demanded by your client in the first place.

[gmaxwell]

So, developers, what's the solution if...
1. You run an exchange where your clients have accounts in your wallet.
2. A malicious client / competing exchange operator makes many small deposits to his account.
3. The malicious client withdrawals all of his money at once and the huge fee gives his account a negative balance, which eats into your reserves.
4. He opens a new account and repeats the process until you're bankrupt and owe your other users their bitcoins back, which you now don't have.
Will any of these solutions work?
- Patch to pay no fees and only allow withdrawals of bitcoins with > X confirmations.
- Institute some convoluted system that disregards small deposits.
- Patch to estimatetxfee and subtract that from withdrawals before sending them.
Or what?

Err.  Bitcoins are fungible, it's pretty much never the case that the coins being withdrawn are at all related to the ones deposited.   Once you have a reasonably large balance you'll be cycling out old coins most of the time unless there is a bank run.

So that mostly just leaves the burden of lots of small inputs, but you'll cycle out the small inputs with other transactions in due time without incurring large fees... but if stuffing your wallet with pennies does become problematic you can simply employ a rule of not crediting dust inputs, or imposing a small fee on deposits.   It doesn't have to be "convoluted". if(amount<0.0001)amount=0;

Some sites, like tradehill for example make you specify the exact input size— and impose significant delays on deposits if you get it wrong. You can reject bitdust transactions at this point too, so no one should be surprised by their dust transaction being lost to them.

There has been some talk about tx fee estimation, but because the coin selection algorithm doesn't guarantee the lowest fees and is perturbed by new inputs it's currently hard to asynchronously estimate fees.

[Paperweight]

What is the minimum deposit size cutoff to be invulnerable to this type of attack? Someone could still dustbin you even with a lot of 1 BTC deposits. They could recycle the same bitcoins over and over.
So now what?
- Use a separate wallet for each user?
- Make a deposit buffer? e.g. Deposits go to one wallet and are sent to the main account wallet in large chunks, minus fees? (Convoluted)
- Join a free transaction relay network?

[gmaxwell]

What is the minimum deposit size cutoff to be invulnerable to this type of attack? Someone could still dustbin you even with a lot of 1 BTC deposits. They could recycle the same bitcoins over and over.
So now what?
- Use a separate wallet for each user?
- Make a deposit buffer? e.g. Deposits go to one wallet and are sent to the main account wallet in large chunks, minus fees? (Convoluted)
- Join a free transaction relay network?

They could also flood you with HTTP requests sent by a million node botnet.  Whats your point?  There is nothing special about bitcoin here, you're also offtopic for this thread as far as I can tell.

FWIW, bitcoin itself resists big coin bitdusting by making the priority depend on tx_value * input_age / tx_size.  So quickly recirculating a large coin doesn't magically get you past it.

[jrmithdobbs]

Actually, in the latest update i fixed the AllowFree() to be normal and only modified bool CreateTransaction(), so that dust spam relaying code becomes unchanged.

Unless I am wrong, the only thing that is changed now is creating new transactions.

Code:

In CreateTransaction():

Before:
                bool fAllowFree = CTransaction::AllowFree(dPriority);
                int64 nMinFee = wtxNew.GetMinFee(1, fAllowFree);

After:
                //Bitcoin NFTF Patch - changed by ShadowOfHarbringer START
                int64 nMinFee = wtxNew.GetMinFee(1, true);
                //Bitcoin NFTF Patch - changed by ShadowOfHarbringer END

So you're willing to put pre-built binaries out there that let people send transactions that wont pass the spam checks. And said binaries also will not forward transactions that don't pass the spam rules if they see them?

You don't see the problem here?

[Paperweight]

What is the minimum deposit size cutoff to be invulnerable to this type of attack? Someone could still dustbin you even with a lot of 1 BTC deposits. They could recycle the same bitcoins over and over.
So now what?
- Use a separate wallet for each user?
- Make a deposit buffer? e.g. Deposits go to one wallet and are sent to the main account wallet in large chunks, minus fees? (Convoluted)
- Join a free transaction relay network?

They could also flood you with HTTP requests sent by a million node botnet.  Whats your point?  There is nothing special about bitcoin here, you're also offtopic for this thread as far as I can tell.

FWIW, bitcoin itself resists big coin bitdusting by making the priority depend on tx_value * input_age / tx_size.  So quickly recirculating a large coin doesn't magically get you past it.

This is the type of problem that ShadowOfHarbringer is trying to solve. A DDOS doesn't bankrupt you and your clients. The values can be randomized. It doesn't have to be done that quickly. Please don't get confrontational.

[grue]

Well, whatever are the details of all this, none of the devs seems to care even to talk to me about this.
So i find this extremely suspicious.
I wonder if some (or all) of them didn't invest heavy money in mining. After all, who would want to intentionally decrease their profits ?

lol, look at all the recent blocks. the transaction fees were super low. take off your tinfoil hat, and admit to yourself that your concern is useless. "derp a dev won't respond to me. MUST BE SOME CONSPIRACY GOING ON INVOLVING THE DEVS, THE ILLUMINATI, AND THE 9001 YEAR OLD MINING CREED"

last 3 blocks:
Generation: 50 + 0.80834234 total fees
Generation: 50 + 0.30386 total fees
Generation: 50 + 0.1656 total fees

[grue]

So, developers, what's the solution if...

1. You run an exchange where your clients have accounts in your wallet.
2. A malicious client / competing exchange operator makes many small deposits to his account.
3. The malicious client withdrawals all of his money at once and the huge fee gives his account a negative balance, which eats into your reserves.
4. He opens a new account and repeats the process until you're bankrupt and owe your other users their bitcoins back, which you now don't have.

Will any of these solutions work?
- Patch to pay no fees and only allow withdrawals of bitcoins with > X confirmations.
- Institute some convoluted system that disregards small deposits.
- Patch to estimatetxfee and subtract that from withdrawals before sending them.

Or what?

is 0.0005 fee on a transaction is too much? There are better ways of sabotaging the system, like getting a bunch of bitcoins, and deleting the wallet.

[gmaxwell]

They could also flood you with HTTP requests sent by a million node botnet.  Whats your point?  There is nothing special about bitcoin here, you're also offtopic for this thread as far as I can tell.

FWIW, bitcoin itself resists big coin bitdusting by making the priority depend on tx_value * input_age / tx_size.  So quickly recirculating a large coin doesn't magically get you past it.

This is the type of problem that ShadowOfHarbringer is trying to solve. A DDOS doesn't bankrupt you and your clients. The values can be randomized. It doesn't have to be done that quickly. Please don't get confrontational.

No he isn't.  It's not a problem that actually exists. You simply charge your clients for the txn fees generated when they are generated. Whats the issue? A DDOS can certainly bankrupt you— it can drive up absolutely astronomic connectivity fees, force you to use commercial mitigation services, and take you offline so that you can't earn the income you need to pay the bills.  Worse, they are hard to stop, compared to simply passing on txn fees to your customers in one form or another where they exist.

I'm not being confrontational, I just didn't follow that you thought this software would be desirable for exchanges simply because no competent exchange is going to run a fork of bitcoin that causes them to get stuck and unspendable outputs in their wallets and makes users wait days or weeks for payments to arrive. Presumably they care enough about the success of bitcoin to not allow this outcome.

[grue]

There are only two lines of code changed, and the change is very minor... So i **seriously** doubt anything could go wrong currently.

Those sound like the famous last words of the Debian OpenSSL package maintainer.

[Paperweight]

It's not a problem that actually exists. You simply charge your clients for the txn fees generated when they are generated. ...simply passing on txn fees to your customers in one form or another where they exist.

If only it was that easy! It also would be a lot easier if fees were a fixed amount per transaction, as grue believes, instead of per KB, but that's not feasible.
The whole problem is that tx fees are added on top of the withdrawal amount. The user's withdrawal transaction proceeds without warning even if the amount+fee is larger than the user's account balance, which depletes your reserves. They just end up with a negative balance. The fees are unpredictable and can be very large - I'm talking 30% as seen in others' posts - when made up of lots of small inputs, even if the inputs are old. You can't charge them the fee if the withdrawal is of their entire balance. Neither can you only allow them to withdraw a small amount at a time because that just compounds the problem.

[grue]

If only it was that easy! It also would be a lot easier if fees were a fixed amount per transaction, as grue believes, instead of per KB, but that's not feasible.

I didn't say it was 0.0005 for all transactions. I meant that most of the time, it was 0.0005

The whole problem is that tx fees are added on top of the withdrawal amount. The user's withdrawal transaction proceeds without warning even if the amount+fee is larger than the user's account balance, which depletes your reserves. They just end up with a negative balance. The fees are unpredictable and can be very large - I'm talking 30% as seen in others' posts - when made up of lots of small inputs, even if the inputs are old. You can't charge them the fee if the withdrawal is of their entire balance. Neither can you only allow them to withdraw a small amount at a time because that just compounds the problem.

this could easily be fixed with a simple patch. somehow integrate the GUI function to determine if a fee is needed to a JSON-RPC method.  besides. this is not a very good way to attack a business.

[Paperweight]

If only it was that easy! It also would be a lot easier if fees were a fixed amount per transaction, as grue believes, instead of per KB, but that's not feasible.

I didn't say it was 0.0005 for all transactions. I meant that most of the time, it was 0.0005

The whole problem is that tx fees are added on top of the withdrawal amount. The user's withdrawal transaction proceeds without warning even if the amount+fee is larger than the user's account balance, which depletes your reserves. They just end up with a negative balance. The fees are unpredictable and can be very large - I'm talking 30% as seen in others' posts - when made up of lots of small inputs, even if the inputs are old. You can't charge them the fee if the withdrawal is of their entire balance. Neither can you only allow them to withdraw a small amount at a time because that just compounds the problem.

this could easily be fixed with a simple patch. somehow integrate the GUI function to determine if a fee is needed to a JSON-RPC method.  besides. this is not a very good way to attack a business.

That would be the way to do it. You would have a loop reducing the amount until amount+fee = intended amount. Can it know for sure what the required .0005 BTC/KB fee will be before sending it?

[gmaxwell]

If only it was that easy! It also would be a lot easier if fees were a fixed amount per transaction, as grue believes, instead of per KB, but that's not feasible.
The whole problem is that tx fees are added on top of the withdrawal amount. The user's withdrawal transaction proceeds without warning even if the amount+fee is larger than the user's account balance, which depletes your reserves. They just end up with a negative balance. The fees are unpredictable and can be very large - I'm talking 30% as seen in others' posts - when made up of lots of small inputs, even if the inputs are old. You can't charge them the fee if the withdrawal is of their entire balance. Neither can you only allow them to withdraw a small amount at a time because that just compounds the problem.

Citation on the 30%? I recall someone here claiming 30% when it was really more like 5% because they failed at math.

It takes about .1816 KB per input, if all inputs are 0.01 and you're paying 0.01 per KB (.20 node behavior) you'll have no more than 18.2% fees for large transactions made of many 0.01 inputs. With 0.0005 BTC/KB it's a 0.9% fee for a big transaction made entirely of 0.01 inputs.  Of course, this goes down very fast when the inputs are larger.

Since acounts are just a daemon local bookeeping function and don't control the inputs, I think it wouldn't be reasonable to change someone a txn fee of more than the base fee just because their TXN got an unlucky selection of coins from the shared inputs.  Instead you should just have a constant fee that covers the average and then some.  It's just a simple cost of doing business.

Of course, an RPC could be provided to query the fees for a TXN in advance (if you can tolerate an approximation, since additional inputs will change it), or a the send could get a maximum fee argument and reject with the required fee if you don't authorize enough.

The txn size related fees (which, FWIW, these patches do _nothing_ to) are simply not going away.  The tx_data has a direct cost on the network— every megabyte stored in the blockchain results in something like 40gigabytes transmitted and stored in the bitcoin network. Without fees for large transactions people will have a good reason do do psycho "backup my data into the blockchain" games and other such stuff.

[Paperweight]

If only it was that easy! It also would be a lot easier if fees were a fixed amount per transaction, as grue believes, instead of per KB, but that's not feasible.
The whole problem is that tx fees are added on top of the withdrawal amount. The user's withdrawal transaction proceeds without warning even if the amount+fee is larger than the user's account balance, which depletes your reserves. They just end up with a negative balance. The fees are unpredictable and can be very large - I'm talking 30% as seen in others' posts - when made up of lots of small inputs, even if the inputs are old. You can't charge them the fee if the withdrawal is of their entire balance. Neither can you only allow them to withdraw a small amount at a time because that just compounds the problem.

Citation on the 30%? I recall someone here claiming 30% when it was really more like 5% because they failed at math.

It takes about .1816 KB per input, if all inputs are 0.01 and you're paying 0.01 per KB (.20 node behavior) you'll have no more than 18.2% fees for large transactions made of many 0.01 inputs. With 0.0005 BTC/KB it's a 0.9% fee for a big transaction made entirely of 0.01 inputs.  Of course, this goes down very fast when the inputs are larger.

Since acounts are just a daemon local bookeeping function and don't control the inputs, I think it wouldn't be reasonable to change someone a txn fee of more than the base fee just because their TXN got an unlucky selection of coins from the shared inputs.  Instead you should just have a constant fee that covers the average and then some.  It's just a simple cost of doing business.

Of course, an RPC could be provided to query the fees for a TXN in advance (if you can tolerate an approximation, since additional inputs will change it), or a the send could get a maximum fee argument and reject with the required fee if you don't authorize enough.

The txn size related fees (which, FWIW, these patches do _nothing_ to) are simply not going away.  The tx_data has a direct cost on the network— every megabyte stored in the blockchain results in something like 40gigabytes transmitted and stored in the bitcoin network. Without fees for large transactions people will have a good reason do do psycho "backup my data into the blockchain" games and other such stuff.

So fees are good! As long as they have to pay a minimum fee of 0.0005 BTC / transaction, even if I set up a very poorly rate-limited exchange...

They send	They pay in fees	You send them	You lose reserves with a negative balance
1 x 100 BTC	0.0005 BTC	1 x 100 BTC	1 Input x 0.1816 KB/Input x 0.0005 BTC/KB = 0.0005 BTC
10 x 10 BTC	0.0050 BTC	1 x 100 BTC	10 Inputs x 0.1816 KB/Input x 0.0005 BTC/KB = 0.0009 BTC
100 x 1 BTC	0.0500 BTC	1 x 100 BTC	100 Inputs x 0.1816 KB/Input x 0.0005 BTC/KB = 0.0091 BTC
1000 x 0.1 BC	0.5000 BTC	1 x 100 BTC	1000 Inputs x 0.1816 KB/Input x 0.0005 BTC/KB = 0.0908 BTC
10,000 x 0.01 BTC	5 BTC	1 x 100 BTC	10,000 Inputs x 0.1816 KB/Input x 0.0005 BTC/KB = 0.9080 BTC
100,000 x 0.001 BTC	50 BTC	1 x 100 BTC	100,000 Inputs x 0.1816 KB/Input x 0.0005 BTC/KB = 9.0800 BTC
1,000,000 x 0.0001 BTC	500 BTC	1 x 100 BTC	1,000,000 Inputs x 0.1816 KB/Input x 0.0005 BTC/KB = 90.8000 BTC

So I guess this attacks isn't so bad because it asymmetrically costs them 5x as many bitcoins in fees as it costs you! Although they can still bankrupt you if they are rich and hate you 

[ShadowOfHarbringer]

Actually, in the latest update i fixed the AllowFree() to be normal and only modified bool CreateTransaction(), so that dust spam relaying code becomes unchanged.

Unless I am wrong, the only thing that is changed now is creating new transactions.

Code:

In CreateTransaction():

Before:
                bool fAllowFree = CTransaction::AllowFree(dPriority);
                int64 nMinFee = wtxNew.GetMinFee(1, fAllowFree);

After:
                //Bitcoin NFTF Patch - changed by ShadowOfHarbringer START
                int64 nMinFee = wtxNew.GetMinFee(1, true);
                //Bitcoin NFTF Patch - changed by ShadowOfHarbringer END

So you're willing to put pre-built binaries out there that let people send transactions that wont pass the spam checks. And said binaries also will not forward transactions that don't pass the spam rules if they see them?

You don't see the problem here?

Unless i overlooked something in the code, CreateTransaction() is not used for relaying transactions, but for creating them (=sending money) is it ?
So what is the problem exactly ?

Well, whatever are the details of all this, none of the devs seems to care even to talk to me about this.
So i find this extremely suspicious.
I wonder if some (or all) of them didn't invest heavy money in mining. After all, who would want to intentionally decrease their profits ?

lol, look at all the recent blocks. the transaction fees were super low. take off your tinfoil hat, and admit to yourself that your concern is useless. "derp a dev won't respond to me. MUST BE SOME CONSPIRACY GOING ON INVOLVING THE DEVS, THE ILLUMINATI, AND THE 9001 YEAR OLD MINING CREED"

last 3 blocks:
Generation: 50 + 0.80834234 total fees
Generation: 50 + 0.30386 total fees
Generation: 50 + 0.1656 total fees

Not all of them are "super low".
There is a topic on this forums with a guy which claims that he was forced to pay something around 20% fees for sending 50 BTC of money (I may remember the exact numbers wrong, but you get the idea).

But that is of course talking about the old version having 0.01 fee, not 0.0005.

[ShadowOfHarbringer]

I would have your client be better and more simple if you are really gonna mess around with txfee. You can get rid of the confusing "this is over the minimum size" message. However, no TX fee = block bloat DOS tool. A naughty boy could scrape the forum and the net for bitcoin addresses and send them all .00000001 thousands of times for $10.

You are talking about a "naughty boy" who cannot change the code himself.

It will still be easy for all technical types/hackers/people with a lot of money to change the mainline client and send as many spam/dust transactions as they want.


-----------
I added a proper warning in the starting post of this topic.
As of now, this fork is not suited for Bitcoin newbies.

[gmaxwell]

I would have your client be better and more simple if you are really gonna mess around with txfee. You can get rid of the confusing "this is over the minimum size" message. However, no TX fee = block bloat DOS tool. A naughty boy could scrape the forum and the net for bitcoin addresses and send them all .00000001 thousands of times for $10.

You are talking about a "naughty boy" who cannot change the code himself.
It will still be easy for all technical types/hackers/people with a lot of money to change the mainline client and send as many spam/dust transactions as they want.

No, a "naughty boy" who can't change the software other people are using. The DDOS protection comes not from the fact that your client won't do it, but from the refusal of most potential neighbors to relay it and miners to miner it. Your client's refusal is your protection from the protection.