[UPDATE: 2015-05-10] Bitcoin Core soft-fork "No Forced TX Fee" v0.10.1 available

[ShadowOfHarbringer]

2011-12-03 Update:

NFTF - version 0.5.0 released.

A fresh tag - nftf-v0.5.0 is avaiable for download.
https://github.com/ShadowOfHarbringer/bitcoin-nftf/tags

Trunk code was also updated:
https://github.com/ShadowOfHarbringer/bitcoin-nftf/tree/

[1713302505]

1713302505

[1713302505]

1713302505

[1713302505]

1713302505

[ShadowOfHarbringer]

2012-01-07 Update:

NFTF - version 0.5.1 released.

A fresh tag - nftf-v0.5.1 is avaiable for download.
https://github.com/ShadowOfHarbringer/bitcoin-nftf/tags

Trunk code has also been merged back:
https://github.com/ShadowOfHarbringer/bitcoin-nftf/tree/

[ShadowOfHarbringer]

2012-02-12 Update:

NFTF - versions 0.5.2 & 0.6.0rc1 released.

NOTE: From now on i will also include release candidate (rc) tags in my fork.

Fresh tags - nftf-v0.5.2, nftf-v0.6.0rc1 are avaiable for download.
https://github.com/ShadowOfHarbringer/bitcoin-nftf/tags

Trunk code has also been merged from mainline client:
https://github.com/ShadowOfHarbringer/bitcoin-nftf/tree/

[finway]

There it is, will this broke the bitcoin transaction_fee_collection_business ?

[gmaxwell]

Since this is back on the top of the forum— it's time for the regular reminder:

This is a terrible patch that puts you at risk and no one should run it.   The overwhelming majority of transactions with the stock reference client do not pay fees, the average _total_ fees per block are about 0.02 BTC— pointing out the lie that miners give a crap about collecting fees from you— the reference client only adds fees to transactions when it would not mine or relay them had they come from someone else, when they are objectively indistinguishable from a DOS attack.  When it does add fees they are usually only 0.0005 BTC.

Recovering from stuck transactions is a major pain and requires careful editing of the wallet binaries.  When a transaction is stuck it may lock up far more than its value.

These patches also don't do what they claim to do— they'll still apply fees in some cases. (Though at least thats a good thing)

It's also the case that no one actively involved and informed in Bitcoin's development is reviewing these patches.  If ShadowOfHarbringer was actually competent do this this work he would not be making misleading claims and would instead be working on code to help users safely recover from stuck transactions.  I would not run code released by him.

[ShadowOfHarbringer]

This is a terrible patch that puts you at risk and no one should run it.   The overwhelming majority of transactions with the stock reference client do not pay fees, the average _total_ fees per block are about 0.02 BTC— pointing out the lie that miners give a crap about collecting fees from you— the reference client only adds fees to transactions when it would not mine or relay them had they come from someone else, when they are objectively indistinguishable from a DOS attack.

Completely untrue, at least it was few months ago (i am not sure if current state is similiar, because i didn't have time to test it lately).
The official client adds fees, even when they are completely unnecessary - meaning they get relayed and confirmed without any fee.

Do some testing before you make claims such as this. I did such tests on my fork with only 2 confirmations (and small sums - like 0.02, 0.10) , and guess what. Fees were NOT necessary. Ever.

Also, there is a proper warning at the start of this topic.

When it does add fees they are usually only 0.0005 BTC.

You miss the point.
I want to have freedom to choose whether i want to pay fees that may or may not be necessary.
This is MY FREEDOM, not developer freedom to decide what to do. This is the exact reason i created this simple patch.

Also, as i already stated somewhere in this topic, **this is not a "proper" fork, just a simple 2 line - patch ** for people who value freedom of choice the same as I do.

The **proper** fork/patch should implement a dialog saying "Are you absolutely sure that you do not want to include fee ? Your money may be lost in the process".

These patches also don't do what they claim to do— they'll still apply fees in some cases. (Though at least thats a good thing)

Of course, it wasn't my intention for the patch to remove ALL fees. It only removes the necessity of paying the fee which started around 0.3.23.
It reverts the client behavior to the one from 0.3.20.

It's also the case that no one actively involved and informed in Bitcoin's development is reviewing these patches.  If ShadowOfHarbringer was actually competent do this this work he would not be making misleading claims and would instead be working on code to help users safely recover from stuck transactions.  

Unfortunately, I do not posess enough C/C++ skill (as i already mentioned above), but i do posess the skills to read&understand C/C++ code (to a certain degree) and do pretty advanced git patchwork.
This is all I am doing here, nothing more is really required to maintain a 2-line patch for God's sake.

I would not run code released by him.

This is your choice. Because you see: It is all about choice. My choice was to create this fork for myself and other people who value individual freedom and are willing to take some risks to achieve that freedom.

[gmaxwell]

Do some testing before you make claims such as this. I did such tests on my fork with only 2 confirmations (and small sums - like 0.02, 0.10) , and guess what. Fees were NOT necessary. Ever.

Yes, and they are also seldom added in those cases. Moreover, The amount of the transaction itself isn't whats relevant for the fee calculations (unless it goes under 0.01 BTC).

I'm just stating a simple fact: The bitcoin software uses the _same_ code to determine if fees are required for relay as it does when adding fees to a transaction.  If it decides to add one, it's only if other nodes wouldn't have relayed (much less mined) the transaction without them.

Yes, I'm sure you tested and found it worked fine. I expect that in those cases the unmodified reference software also would not have applied a fee.

I want to have freedom to choose whether i want to pay fees that may or may not be necessary.
[...]
Also, as i already stated somewhere in this topic, **this is not a "proper" fork, just a simple 2 line - patch ** for people who value freedom of choice the same as I do.

And freedom you have, there is nothing I could do to take that away from you even if I wanted to, though I don't.
Your promotion on your signature, "Getting robbed by miners", however isn't just about providing a choice. As far as I can tell it's about spreading misinformation, fear, uncertainty, and doubt.  It reduces choice by clouding people's judgement with the fear that they'll lose money and may cause them to run your fork when under a rational analysis they would not.

Your change is also not a 2 line patch. Diffing your RC tag and the reference client RC tag gives me a 289 line diff.  You appear to be missing at least one random patch from mainline in December. I'm sure this was a simple mistake, but what happens when such a mistake introduces a data corruption bug?

In any case, I didn't post for your benefit, we've gone over this before.  I was just making a regular cautionary post so that people who are just seeing this thread for the first time won't jump to the end and miss the earlier discussion.

[ShadowOfHarbringer]

Yes, and they are also seldom added in those cases. Moreover, The amount of the transaction itself isn't whats relevant for the fee calculations (unless it goes under 0.01 BTC).

I'm just stating a simple fact: The bitcoin software uses the _same_ code to determine if fees are required for relay as it does when adding fees to a transaction.  If it decides to add one, it's only if other nodes wouldn't have relayed (much less mined) the transaction without them.

Yes, I'm sure you tested and found it worked fine. I expect that in those cases the unmodified reference software also would not have applied a fee.

Actually, I did a 2-level testing.
First, i tested the "normal" client to check if it adds fees. Then i did an identical transaction with identical sum of money.
And guess what. The mainline client ALWAYS (state as of december 2011) wants a fee when you resend money that are younger than 30 minutes.

My client does not. And the transaction produced by my client got confirmed easily (up to 2 hours).

I want to have freedom to choose whether i want to pay fees that may or may not be necessary.
[...]
Also, as i already stated somewhere in this topic, **this is not a "proper" fork, just a simple 2 line - patch ** for people who value freedom of choice the same as I do.

And freedom you have, there is nothing I could do to take that away from you even if I wanted to, though I don't.
Your promotion on your signature, "Getting robbed by miners", however isn't just about providing a choice. As far as I can tell it's about spreading misinformation, fear, uncertainty, and doubt. 

Well, actually i think you may be at least partially right here.
I will remove the "getting robbed by miners" ad, and replace it with something "lighter".

It reduces choice by clouding people's judgement with the fear that they'll lose money and may cause them to run your fork when under a rational analysis they would not.

Your change is also not a 2 line patch. Diffing your RC tag and the reference client RC tag gives me a 289 line diff.
  You appear to be missing at least one random patch from mainline in December. I'm sure this was a simple mistake, but what happens when such a mistake introduces a data corruption bug?

This is interesting.
Gitk shows that my branch is fully merged with master, except for my little patch.

I will diff it myself now and show you the results in a few minutes.

In any case, I didn't post for your benefit, we've gone over this before.  I was just making a regular cautionary post so that people who are just seeing this thread for the first time won't jump to the end and miss the earlier discussion.

There is a "cautionary" big, red warning in the first post, shouldn't that be enough ?

[gmaxwell]

Actually, I did a 2-level testing.
First, i tested the "normal" client to check if it adds fees. Then i did an identical transaction with identical sum of money.
And guess what. The mainline client ALWAYS (state as of december 2011) wants a fee when you resend money that are younger than 30 minutes.

My client does not. And the transaction produced by my client got confirmed easily (up to 2 hours).

Ah.

What you're having happen there is that when you send your transaction is dropped by the network— lacking the required fees because it looks like a DOS attack (fast turnaround)—, but after every block that doesn't contain your transaction the client tries retransmitting it (with a random delay).  After two hours or so, the input coins have matured enough that they no longer require a fee.  After that when your client resends then it goes through.

If you'd just waited the same time it would have sent without a fee in mainline.  From a usability perspective, when the times are reasonably short, it might be better to wait like that... but if you shut down right after sending the transaction will be stuck until you start back up and wait an hour or so, and a lot of people do that.

I've had transaction queuing on my feature wishlist for a while (mostly for the purpose of automatically grouping transactions into a sendmany). Perhaps that would be a good way to address this.

FWIW, you can do this testing with Testnet too. So you can do a bunch without having to move around a bunch of real bitcoin.

There is a "cautionary" big, red warning in the first post, shouldn't that be enough ?

I don't know about you, but I usually skip to the end of long threads and don't bother reading much of the start.

[ShadowOfHarbringer]

@gmaxwell

OK, seems you were right. Thanks for pointing that out, i could not notice it myself.

Some "weird magic" happened during my git patchwork, and some conflicts didn't merge properly.
The errors were so serious though that the program would probably not compile (at least i hope so). Lucky the changes were only visible for a couple of hours.

Updated & fixed tags are now avaiable, the old ones were removed. Everything is now merged properly, including the trunk.

Also, from now on i will always diff all tags & trunk with mainline client in case of some more merge errors.

[ShadowOfHarbringer]

Ah.

What you're having happen there is that when you send your transaction is dropped by the network— lacking the required fees because it looks like a DOS attack (fast turnaround)—, but after every block that doesn't contain your transaction the client tries retransmitting it (with a random delay).  After two hours or so, the input coins have matured enough that they no longer require a fee.  After that when your client resends then it goes through.

If you'd just waited the same time it would have sent without a fee in mainline.  From a usability perspective, when the times are reasonably short, it might be better to wait like that... but if you shut down right after sending the transaction will be stuck until you start back up and wait an hour or so, and a lot of people do that.

You are absolutely right, however still missing the point.

It is my risk to take and whether i still want to gamble and send the coins should be my decision.
This is why i bugged mainline devs multiple times for a simple configuration setting or a simple "are you sure" dialog, or both.

They are acting like the issue doesn't exist at all. Why is a simple configuration setting such a problem ? Open source is all about choice.

[dani147624]

These patches also don't do what they claim to do— they'll still apply fees in some cases. (Though at least thats a good thing)

Of course, it wasn't my intention for the patch to remove ALL fees. It only removes the necessity of paying the fee which started around 0.3.23.
It reverts the client behavior to the one from 0.3.20.

I'm not quite happy with that, why not eliminate forced transaction fees altogether?

Here's the patch I use:

Code:

--- main.h 2012-02-18 14:00:22.008162091 +0100
+++ main.h 2012-02-18 14:00:22.008162091 +0100
@@ -35,8 +35,8 @@
 static const int MAX_BLOCK_SIGOPS = MAX_BLOCK_SIZE/50;
 static const int64 COIN = 100000000;
 static const int64 CENT = 1000000;
-static const int64 MIN_TX_FEE = 50000;
-static const int64 MIN_RELAY_TX_FEE = 10000;
+static const int64 MIN_TX_FEE = 0; // Changed by dani147624
+static const int64 MIN_RELAY_TX_FEE = 0; // Changed by dani147624
 static const int64 MAX_MONEY = 21000000 * COIN;
 inline bool MoneyRange(int64 nValue) { return (nValue >= 0 && nValue <= MAX_MONEY); }
 static const int COINBASE_MATURITY = 100;
@@ -568,6 +568,8 @@
 
     int64 GetMinFee(unsigned int nBlockSize=1, bool fAllowFree=true, enum GetMinFee_mode mode=GMF_BLOCK) const
     {
+ // Changes by dani147624: this should now check what the minimum fee would be, and then return 0 anyway.
+
         // Base fee is either MIN_TX_FEE or MIN_RELAY_TX_FEE
         int64 nBaseFee = (mode == GMF_RELAY) ? MIN_RELAY_TX_FEE : MIN_TX_FEE;
 
@@ -602,13 +604,14 @@
         if (nBlockSize != 1 && nNewBlockSize >= MAX_BLOCK_SIZE_GEN/2)
         {
             if (nNewBlockSize >= MAX_BLOCK_SIZE_GEN)
-                return MAX_MONEY;
+                return 0; // Changed by dani147624
+
             nMinFee *= MAX_BLOCK_SIZE_GEN / (MAX_BLOCK_SIZE_GEN - nNewBlockSize);
         }
 
         if (!MoneyRange(nMinFee))
             nMinFee = MAX_MONEY;
-        return nMinFee;
+        return 0; // Changed by dani147624
     }

I'm waiting for a transaction created with this to be confirmed, but have had little luck in the last 2 days. It's input has many transactions (maybe 20-40) of newly generated coins (from p2pool) and it's output has 1 BTC plus some change. I hope it will get included in a block once. (It would take quite some months for me to mine it with my 160 MH/s.) Details of transaction:

Code:

"account" : "",
        "address" : "1AaTGqfaGhuqUq8Eaq6dmLsDjEFjtwGkQA",
        "category" : "send",
        "amount" : -1.00000000,
        "fee" : 0.00000000,
        "confirmations" : 0,
        "txid" : "ded01bdef405f10258d4bb180f79680f83023e8c179d3e165f4a2db268fa34fb",
        "time" : 1329412273

I did manage to get about a thousand 1 satoshi, no fee transactions into the testnet blockchain with it, but I had to mine the block myself.

[westkybitcoins]

These patches also don't do what they claim to do— they'll still apply fees in some cases. (Though at least thats a good thing)

Of course, it wasn't my intention for the patch to remove ALL fees. It only removes the necessity of paying the fee which started around 0.3.23.
It reverts the client behavior to the one from 0.3.20.

I'm not quite happy with that, why not eliminate forced transaction fees altogether?

Because some fees are mandatory, and the network rejects transactions without those fees as being spam.

I suspect the transaction you mentioned above won't ever be confirmed.

[ShadowOfHarbringer]

These patches also don't do what they claim to do— they'll still apply fees in some cases. (Though at least thats a good thing)

Of course, it wasn't my intention for the patch to remove ALL fees. It only removes the necessity of paying the fee which started around 0.3.23.
It reverts the client behavior to the one from 0.3.20.

I'm not quite happy with that, why not eliminate forced transaction fees altogether?

Because some fees are mandatory, and the network rejects transactions without those fees as being spam.

I suspect the transaction you mentioned above won't ever be confirmed.

Yes, exactly.
The only mandatory fees that are not-really-so-mandatory are the ones which were introduced around 0.3.23.

The first conception of my fork was to remove necessity of any fee. But removing other requirements can be IS dangerous (network would not relay such transacactions or other weird errors could happen) so i dumped that idea.

Code:

(...)

A suggestion - If you want to be really useful, try coding a proper dialog like "Are you sure you really want not to include a fee ? Transaction may never be confirmed !" or/and a configuration setting in the bitcoin.conf file, so that power users/people who know what they are doing can send whatever transaction they want to.

Right now, this fork is not a "proper" solution. With a patch like that, it could become good enough to be pulled into the main bitcoin tree perhaps.

[etotheipi]

Shadow,

I wanted to follow up on a recent conversation I had with you concerning Armory and transaction fees. 

I made a mistake in my initial assessment of what I could do with it.  I have implemented all the tx fee calculations and exactly when they need to be applied.  The default behavior of Armory is to let you select 0.0 fee, but if the calculation determines you need more, you either have to increase the fee, or cancel.

I had decided to put in an "Override Minimum Fee" setting that you would have to change by hand, but it could be done.  This was all implemented and the dialogs are even created and work.  The problem is that since Armory connects through localhost-Satoshi-client, transactions that "need" a fee but don't have them, will not get forwarded.  Thus, even though the network is full of nodes that will take zero-fee txs, Armory only has one peer, and the forced-zero-fee transactions are DOA. 

It seems, the only way for Armory to do this, is if this fork is used along with the over-ride min fee option (though, I haven't tried it).   In the future, I will have some kind of independent networking that will allow me to sidestep this problem.  Until then, I can't live up to my promise of supporting forced zero-tx fees! 

[ShadowOfHarbringer]

It seems, the only way for Armory to do this, is if this fork is used along with the over-ride min fee option (though, I haven't tried it).   In the future, I will have some kind of independent networking that will allow me to sidestep this problem.  Until then, I can't live up to my promise of supporting forced zero-tx fees! 

I actually made this fork to save "the old way" of doing transactions, so that it will not be forgotten.
You can use my code to implement the override properly, as the only thing it does (2 lines of code changed) is it reverts the client behavior to pre-0.3.23 one.

It should be extremely easy to add an extra configuration switch/parameter/whatever, so you can choose between default Satoshi-client-behavior mode, or NFTF-client-behavior mode when creating a transaction.

I am happy to see that my fork can be actually useful for something important.

[btc_artist]

Shadow,

I wanted to follow up on a recent conversation I had with you concerning Armory and transaction fees. 

I made a mistake in my initial assessment of what I could do with it.  I have implemented all the tx fee calculations and exactly when they need to be applied.  The default behavior of Armory is to let you select 0.0 fee, but if the calculation determines you need more, you either have to increase the fee, or cancel.

I had decided to put in an "Override Minimum Fee" setting that you would have to change by hand, but it could be done.  This was all implemented and the dialogs are even created and work.  The problem is that since Armory connects through localhost-Satoshi-client, transactions that "need" a fee but don't have them, will not get forwarded.  Thus, even though the network is full of nodes that will take zero-fee txs, Armory only has one peer, and the forced-zero-fee transactions are DOA. 

It seems, the only way for Armory to do this, is if this fork is used along with the over-ride min fee option (though, I haven't tried it).   In the future, I will have some kind of independent networking that will allow me to sidestep this problem.  Until then, I can't live up to my promise of supporting forced zero-tx fees! 

You could work around it though.  When you have a forced zero-fee tx, instead of just passing the tx to the local bitcoind, send a getaddr message to the local bitcoind, parse the addresses received, then connect to all of those nodes and broadcast the tx to them.

If I understand correctly, you need the local bitcoind for the blockchain, but there's no reason you have to use it exclusively for broadcasting transactions.

[ShadowOfHarbringer]

You could work around it though.  When you have a forced zero-fee tx, instead of just passing the tx to the local bitcoind, send a getaddr message to the local bitcoind, parse the addresses received, then connect to all of those nodes and broadcast the tx to them.

If I understand correctly, you need the local bitcoind for the blockchain, but there's no reason you have to use it exclusively for broadcasting transactions.

This might work, but using NFTF fork would probably be much easier & quickier solution.

[btc_artist]

The quickest and easiest solution is probably to change the coin selection algorithm to always select the oldest coin(s) possible for the inputs, thereby not triggering a fee in bitcoind.  In most/many cases that will avoid the fee.

[ShadowOfHarbringer]

The quickest and easiest solution is probably to change the coin selection algorithm to always select the oldest coin(s) possible for the inputs, thereby not triggering a fee in bitcoind.  In most/many cases that will avoid the fee.

"Most cases" is still not satisfactory for me, hence the fork.
I like my freedom served fresh & juicy on a plate.

When the mainline client gets a proper "are you sure" dialog and/or a configuration setting "Always.AllowNoFees.IKnowWhatIamDoing = 0/1" implemented, this fork will become obsolete. Since then, i will continue maintaining it.