[bitcoind] Transaction fees: One annoying shit i have stumbled upon

[ShadowOfHarbringer]

Hello,

While developing a bitcoin-related app, i have stumbled upon some seriously annoying shit.

The problem is that apparently, using the default client's bitcoind, there is currently no way to determine what the transaction fee is going to be, before sending money ! (So i could, for example, tell the user how much will he be paying BEFORE he sends the money).

Should there be a method in the API like "checksendtransactionfees <address> <amount>" ?

Can somebody correct me please if am I wrong here?

[2112]

Should there be a method in the API like "checksendtransactionfees <address> <amount>" ?

Wouldn't work. The knapsack problem solver user to select coins is stochastic (uses rand()), therefore the coins selected from the wallet are really random for each call.

You can probably propose a change where a rand() is replaced with some pseudo-random algorithm that seeds of some explicit value and therefore is 100% reproducible.

Satoshi bitcoin client is a dream thing to pull the accounting fraud.

[Pieter Wuille]

There used to be a patch or pull request that added the ability to create a transaction for preview, i.e. not transmitting it yet to the network. This transaction could then be inspected, and afterwards either confirmed or aborted.

I very much liked that idea as a solution to the unknown-fees problem, but I can't find it anymore.

[kjj]

The fee is miniscule.  Just eat it and call it a cost of doing business.

[2112]

The fee is miniscule.  Just eat it and call it a cost of doing business.

kjj probably runs a mining pool which collects those fees and doesn't distribute them to the actual miners. So your cost will be his gain.

Isn't the bitcoin ecosystem wonderfully egalitarian that way?

[ShadowOfHarbringer]

The fee is miniscule.  Just eat it and call it a cost of doing business.

kjj probably runs a mining pool which collects those fees and doesn't distribute them to the actual miners. So your cost will be his gain.

Isn't the bitcoin ecosystem wonderfully egalitarian that way?

Actually, i am running my own fork of Bitcoin, which does not enforce fees.
https://bitcointalk.org/index.php?topic=22434.0;all
https://github.com/ShadowOfHarbringer/bitcoin-nftf/

I just wanted to know, if there is a way to do it without using modified clients.

[ShadowOfHarbringer]

Should there be a method in the API like "checksendtransactionfees <address> <amount>" ?

Wouldn't work. The knapsack problem solver user to select coins is stochastic (uses rand()), therefore the coins selected from the wallet are really random for each call.

You can probably propose a change where a rand() is replaced with some pseudo-random algorithm that seeds of some explicit value and therefore is 100% reproducible.

That is not a problem really.

The method i proposed could work the following way, by "preparing" the transaction:

1. Make the rand() and prepare the transaction + calculate fees
2. The transaction is stored in a "prepared" or "locked" state - all bitcoins from the transaction are locked, until the transaction is unlocked by user.
3. If the user likes the fees, he confirms the transaction, it becomes unlocked, and it is then relayed to the network.

This shouldn't be extremely hard to implement.

[2112]

This shouldn't be extremely hard to implement.

Famous last words. Garbage collecting stale locks on the wallet is a nice gig, you'll spend endless nights debugging it.

[ShadowOfHarbringer]

This shouldn't be extremely hard to implement.

Famous last words. Garbage collecting stale locks on the wallet is a nice gig, you'll spend endless nights debugging it.

Perhaps.

Unfortunately I don't do advanced C++ programming, so I cannot do it myself anyway.
Also there seems to be a lot of unwillingness among current developers for working on any solution to this problem.

That however does not surprise me, as the major devs probably invested heavy money into mining rigs, so why would they damage their business ?

An easy & quick solution could be for example, to return error if transaction fees are set to 0.00, and requested transaction requires a fee.

But obvious mining cartel is obvious.

[Pieter Wuille]

Should there be a method in the API like "checksendtransactionfees <address> <amount>" ?

Wouldn't work. The knapsack problem solver user to select coins is stochastic (uses rand()), therefore the coins selected from the wallet are really random for each call.

You can probably propose a change where a rand() is replaced with some pseudo-random algorithm that seeds of some explicit value and therefore is 100% reproducible.

That is not a problem really.

The method i proposed could work the following way, by "preparing" the transaction:

1. Make the rand() and prepare the transaction + calculate fees
2. The transaction is stored in a "prepared" or "locked" state - all bitcoins from the transaction are locked, until the transaction is unlocked by user.
3. If the user likes the fees, he confirms the transaction, it becomes unlocked, and it is then relayed to the network.

This shouldn't be extremely hard to implement.

As I said, it already was implemented: https://github.com/TheBlueMatt/bitcoin/commit/5350a907a57b88bd86827c1f3e41b571cf130401. It seems to have been closed awaiting rebase...

[ShadowOfHarbringer]

Should there be a method in the API like "checksendtransactionfees <address> <amount>" ?

Wouldn't work. The knapsack problem solver user to select coins is stochastic (uses rand()), therefore the coins selected from the wallet are really random for each call.

You can probably propose a change where a rand() is replaced with some pseudo-random algorithm that seeds of some explicit value and therefore is 100% reproducible.

That is not a problem really.

The method i proposed could work the following way, by "preparing" the transaction:

1. Make the rand() and prepare the transaction + calculate fees
2. The transaction is stored in a "prepared" or "locked" state - all bitcoins from the transaction are locked, until the transaction is unlocked by user.
3. If the user likes the fees, he confirms the transaction, it becomes unlocked, and it is then relayed to the network.

This shouldn't be extremely hard to implement.

As I said, it already was implemented: https://github.com/TheBlueMatt/bitcoin/commit/5350a907a57b88bd86827c1f3e41b571cf130401. It seems to have been closed awaiting rebase...

Nice !

However I will not be using an untested code in prod evironment.

I hope it will be added to mainline client in the future.

[kjj]

The fee is miniscule.  Just eat it and call it a cost of doing business.

kjj probably runs a mining pool which collects those fees and doesn't distribute them to the actual miners. So your cost will be his gain.

Isn't the bitcoin ecosystem wonderfully egalitarian that way?

Actually, no, I don't run a pool.  And my mining rigs are almost always on btcguild, which doesn't distribute fees to members, so I have nothing to gain directly by this.

But finding fees in advance is only possible if you accept some horrible preconditions.  If you are willing to lock parts of the wallet in advance and accept the proposed fees before the next block is found, you can do it.  Pay attention to the second half of that.  If you ignore it, you are absolutely certain to be paying too much in fees over time.

I really like the way flexcoin handles this problem.  They charge a predetermined fee when a user wants to send, and the difference (in aggregate) between the fee they charge and the fee they pay to the bitcoin network is distributed as profit to them and as a dividend "refund" to their depositors.

Oh, and if anyone seriously believes that the devs are obfuscating the mining fees intentionally because they profit from it, you are bat-shit crazy, and I just thought you should know that.

[2112]

Actually, no, I don't run a pool.  And my mining rigs are almost always on btcguild, which doesn't distribute fees to members, so I have nothing to gain directly by this.
[...]
Oh, and if anyone seriously believes that the devs are obfuscating the mining fees intentionally because they profit from it, you are bat-shit crazy, and I just thought you should know that.

Well, I apologise for being presumptious about your personal involvement in the fee-skimming scam. Its my personal bias I'm getting from the bitcoin promoters that I personally met.

As to who is bat-shit crazy it will be probably for the courts to decide. This pertains especially to those developers who reside in the USA and other coutries with an adversarial legal system. The various fee-skimming scams are just such an juicy target for a class action lawsuit in the Eastern Texas Circuit that somebody there will just grab this opportunity.

It either that or bitcoin will tank so low that not even a bottom-feeding lawyer will be willing to get out of the bed.

[ShadowOfHarbringer]

Actually, no, I don't run a pool.  And my mining rigs are almost always on btcguild, which doesn't distribute fees to members, so I have nothing to gain directly by this.
[...]
Oh, and if anyone seriously believes that the devs are obfuscating the mining fees intentionally because they profit from it, you are bat-shit crazy, and I just thought you should know that.

Well, I apologise for being presumptious about your personal involvement in the fee-skimming scam. Its my personal bias I'm getting from the bitcoin promoters that I personally met.

As to who is bat-shit crazy it will be probably for the courts to decide.

I think that involving lawyers is not so necessary.

This is an open-source project, everybody can write their own client. Also, it is a choice of the collective what fees will be enforced. If too many people will be annoyed by fee-skimming, they will move to forked clients, such as mine.

The reasons why people are not rioting against it are probably
1) Larger fees do protect the network from penny flooding attacks
2) This is not annoying enough (yet).

[2112]

The reasons why people are not rioting against it are probably
1) Larger fees do protect the network from penny flooding attacks
2) This is not annoying enough (yet).

3) Lawyers have sufficient income from other sources. The bitcoin targets are not yet ripe enough. By "ripe" I mean that nobody yet signed off on their audits as "true & correct". Once the word gets out among the lawyers that bitcoin-using businesses routinely sign off the perjurous audits that don't follow the GAAP the writs will start flying.

[kjj]

As to who is bat-shit crazy it will be probably for the courts to decide. This pertains especially to those developers who reside in the USA and other coutries with an adversarial legal system. The various fee-skimming scams are just such an juicy target for a class action lawsuit in the Eastern Texas Circuit that somebody there will just grab this opportunity.

It either that or bitcoin will tank so low that not even a bottom-feeding lawyer will be willing to get out of the bed.

What fee-skimming scams?  Are we still talking about network transaction fees?

[ShadowOfHarbringer]

As to who is bat-shit crazy it will be probably for the courts to decide. This pertains especially to those developers who reside in the USA and other coutries with an adversarial legal system. The various fee-skimming scams are just such an juicy target for a class action lawsuit in the Eastern Texas Circuit that somebody there will just grab this opportunity.

It either that or bitcoin will tank so low that not even a bottom-feeding lawyer will be willing to get out of the bed.

What fee-skimming scams?  Are we still talking about network transaction fees?

No, we are talking about not giving user the choice whether he/she should risk his/her money and send the bitcoins without any fee.

Also, check out this topic: https://bitcointalk.org/index.php?topic=22434.0;all

BTW,
Today i did a small test. I re-sent 0.10 BTC having only 2 confirmations to another address, without paying a fee (using my modified client). The resulting transaction was low priority.

Guess what. It was sent without any problems, got 6 confirmations after about 2 hours.

So I wouldn't say that forced transaction fees are so needed.

[2112]

What fee-skimming scams?  Are we still talking about network transaction fees?

Mining pools mostly. Out of the top 10 is there any that doesn't skim or is upfront about the fact that it skims the "means to assure the security of the network transactions".

Also, to the lesser extent, the clients with a bug "change is to small to return it to the wallet, so send it to the miners." It was re-introduced in 0.3.24, which is the current official one.

It also pertains to you:

The fee is miniscule.  Just eat it and call it a cost of doing business.

What if your or ShadowOfHarbinger's software were used to run somebody's else business? You can eat your own fees just fine, but if you make someone else "eat fees" that is somewhere between "negligent accounting" and "conspiracy to commit fraud in accounting". The last one is when someone modifies the software that handles the transactions. In bitcoin ecosystem pretty much everyone runs modified software, so lawyers will have a field day in wringing the settlements from the naive programmers and business-people who hired them.

[ShadowOfHarbringer]

Also, to the lesser extent, the clients with a bug "change is to small to return it to the wallet, so send it to the miners." It was re-introduced in 0.3.24, which is the current official one.

Can you please point me to the exact source (line of code) ?
I will also remove it from my fork.

I didn't know that fee-skimming scams got so serious these days.

[Pieter Wuille]

The "change is too small to go to wallet, let's send it to the miner" thing on itself wasn't a bug.

The problem was that 0.3.24 reintroduced a bug where that calculation went wrong, and occasionally up to 0.01 BTC would be sent as fee (instead of 0.0005 BTC). It's fixed in 0.4, by the way.