Bitcoin Forum
March 28, 2024, 06:44:15 PM *
News: Latest Bitcoin Core release: 26.0 [Torrent]
 
   Home   Help Search Login Register More  
Pages: « 1 [2] 3 »  All
  Print  
Author Topic: [BOUNTY] A patch for bitcoind to modify tx list in "getmemorypool"  (Read 5409 times)
Matt Corallo
Hero Member
*****
Offline Offline

Activity: 755
Merit: 515


View Profile
March 26, 2012, 05:35:10 PM
 #21

Thats it.  Although 0.01 is somewhat arbitrarily chosen by me.  We need to start somewhere. I feel the service provided is worth AT LEAST 1 bitcent and it is a nice round value.   The larger goal is to allow ALL miners (not just me) to see fees at a level they feel aproporiate.
I think everyone would like to see that, but  something slightly more complicated and thorough needs to be thought up.

If anyone has an issue with 0.01 BTC as a fee amount it is better discussed in another thread on appropriate fee pricing.  The code would allow enforcing fees on as low as flat 1 satoshi per tx (of unlimited size - although soon you would hit the spam limit).
No, actually it wouldnt.  This patch does not change the minimum default fee in AcceptToMemoryPool which is required for a tx to a. get relayed through the network and b. be accepted by your node.

I don't want to clutter this thread up too much but from this humble begining it would be my hope to see fee handling become more robust over time.   Bitcoin will eventually require fees to support at least part of the network cost.  Miners will need robust tools to price their service appropriately:
Agreed, the fee system in Bitcoin as it stands now is not very well done.  In fact, its quite poor.  However, a simple change like this really doesnt help much.  Though individual miners who mine in a pool and thus don't make much on txfees couldnt care less whether a particular tx with a low fee is accepted, large-scale miners who want as many coins as possible have absolutely no incentive to increase mintxfee.  Though this only partially applies today (as the 50 BTC per block is usually much higher than any fees), in the future as the 50 BTC/block subsidy decreases, any incentive miners had to do this "for the good of the network" goes away and it becomes get a few coins, or get nothing.  Thus, the fee system could use some help to avoid this very dilemma.

Bitcoin Core, rust-lightning, http://bitcoinfibre.org etc.
PGP ID: 07DF 3E57 A548 CCFB 7530  7091 89BB B866 3E2E65CE
1711651455
Hero Member
*
Offline Offline

Posts: 1711651455

View Profile Personal Message (Offline)

Ignore
1711651455
Reply with quote  #2

1711651455
Report to moderator
1711651455
Hero Member
*
Offline Offline

Posts: 1711651455

View Profile Personal Message (Offline)

Ignore
1711651455
Reply with quote  #2

1711651455
Report to moderator
"If you don't want people to know you're a scumbag then don't be a scumbag." -- margaritahuyan
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction.
1711651455
Hero Member
*
Offline Offline

Posts: 1711651455

View Profile Personal Message (Offline)

Ignore
1711651455
Reply with quote  #2

1711651455
Report to moderator
1711651455
Hero Member
*
Offline Offline

Posts: 1711651455

View Profile Personal Message (Offline)

Ignore
1711651455
Reply with quote  #2

1711651455
Report to moderator
1711651455
Hero Member
*
Offline Offline

Posts: 1711651455

View Profile Personal Message (Offline)

Ignore
1711651455
Reply with quote  #2

1711651455
Report to moderator
DeathAndTaxes (OP)
Donator
Legendary
*
Offline Offline

Activity: 1218
Merit: 1063


Gerald Davis


View Profile
March 26, 2012, 06:07:26 PM
 #22

I think everyone would like to see that, but  something slightly more complicated and thorough needs to be thought up.

Of course but right now you choices are accept all tx or pocess no tx.  Here at least you now have some crude control.  This was intended to be a starting point to a) provide p2pool users some control and b) spark a conversation on mining fees.

Quote
No, actually it wouldnt.  This patch does not change the minimum default fee in AcceptToMemoryPool which is required for a tx to a. get relayed through the network and b. be accepted by your node.

For most tx the default fee is 0.00.  It is based on coinage (1 day, 1 BTC input = free).  One could reject only zero fee txs (accept tx which don't violate spam rules and have at least 1 satoshi in fees).

One could modify the spam rules but I didn't want that because I think it is reckless.  The spam rules are designed not to cost users money but to protect the network.  If enough miners foolishly removed the spam rules one could cripple the network, users, wallets with massive (TBs) worth of spam for a negigible cost.  I intentionally asked for the spam rules to be enforced for that reason. 


Quote
Though this only partially applies today (as the 50 BTC per block is usually much higher than any fees), in the future as the 50 BTC/block subsidy decreases, any incentive miners had to do this "for the good of the network" goes away and it becomes get a few coins, or get nothing.  Thus, the fee system could use some help to avoid this very dilemma.

True miners will reduce their direct compensation by excluding paying low fee tx but the amount they lose is essentially negligible.   Chart tx vs fees in the blockchain for the last 100,00 or so tx.  You will notice that even excluding tx w/ fee < 0.01 results in only a negligible drop in gross revenue (as in pennies a block).  With a lower min fee you "lose" even less.

At this point it is more the principal of the issue and to spark a debate.  The "cost" to a miner is essentially nothing (even one with dozens of GH/s).  If not enough miners start enforcing higher fees then fees won't rise but the amount lost remains "almost nothing".  If enough miners start enforcing higher fees then all miners benefit.  Yeah there is some aspect of tragedy of the commons but w/ block subsidies still high (yes 25 BTC is high) the "cost" of helping is nearly nothing.
DeathAndTaxes (OP)
Donator
Legendary
*
Offline Offline

Activity: 1218
Merit: 1063


Gerald Davis


View Profile
March 26, 2012, 06:15:47 PM
 #23

Looks good to me.  Thanks for the prompt response.
If this is complete, where would you like me to send a refund (if you want one) of your advance? Let me know if you would prefer to wait for testing and merging to mainline (which might require more time, depending on the reactions of other developers).

Polvos, you also offered to contribute toward the bounty; if you feel the desire to tip me, you can use 1DGzpZzce1c7nsg1SN7exV6bsaju1Mcrc6 - it's fine with me if you choose not to, though.

I've submitted an upstream pull request. Miners who want this feature should express their support on this thread, and developers with comments can do so on the pull request itself.

No keep the advance.  We likely need some testing and the code may require some tweaking.  20 BTC IMHO is well spent. 

Now is there a guide anywhere for compiling modified version of bitcoind under windows?
Matt Corallo
Hero Member
*****
Offline Offline

Activity: 755
Merit: 515


View Profile
March 26, 2012, 06:35:32 PM
 #24

Of course but right now you choices are accept all tx or pocess no tx.  Here at least you now have some crude control.  This was intended to be a starting point to a) provide p2pool users some control and b) spark a conversation on mining fees.
Right now, the choice for those using the original satoshi client is either a) accept everything that passes the anti-spam rules or b) dont mine.
Don't get me wrong, this patch is a cool idea, but in the long run its unlikely to make a difference, and in the short run, you really need to get mining pools involved, a few p2pool users doesnt do much...
To spark a discussion on txfees, you need to come up with a real solution to the long term issues with txfees, which this is not.

At this point it is more the principal of the issue and to spark a debate.  The "cost" to a miner is essentially nothing (even one with dozens of GH/s).  If not enough miners start enforcing higher fees then fees won't rise but the amount lost remains "almost nothing".  If enough miners start enforcing higher fees then all miners benefit.  Yeah there is some aspect of tragedy of the commons but w/ block subsidies still high (yes 25 BTC is high) the "cost" of helping is nearly nothing.
Again, this isnt a serious suggestion, nor is it actually a suggestion at all to modify/fix the fee system, it just makes it easier for people to do what was already being done by some pools/miners.  Even if miners decide to start increasing minfee now during high BTC subsidy, all they do is force users to pay more fee now for relatively no gain to miners.  When the subsidy falls low enough, miners would go back to what is in their individual best interest, which is to accept 1-satoshi-fee txes. 

Bitcoin Core, rust-lightning, http://bitcoinfibre.org etc.
PGP ID: 07DF 3E57 A548 CCFB 7530  7091 89BB B866 3E2E65CE
Haplo
Full Member
***
Offline Offline

Activity: 168
Merit: 100



View Profile
March 26, 2012, 07:23:12 PM
 #25

".01 BTC" is not a reasonable fee, period. Here's why.

I lend 1BTC to someone for 1 day, interest paid is 1%, total 1.01 BTC due.

I pay him the 1BTC loan, there's 0.01BTC paid to you.

He spends his 1BTC on whatever it is he wanted, there's 0.01 BTC paid to you.

He trades cash for 1.01BTC to pay me back, another 0.01 BTC paid to you.

He sends me the 1.01BTC, there's another .01BTC paid to you.

The interest on the loan was 0.01 BTC, total interest paid to me = 0.01BTC, interest paid to you just for the tx, .04 BTC, or 4 times the interest the loan was worth. This sort of 1BTC/1% transaction happens regularly on the lending forum.

I should point out that western union does indeed charge based on how much you send, although it's a flat rate for up to a certain amount, but that's pretty irrelevant. I would say a "Fair" rate is more like 0.05%, maybe 0.025%, with a spam fee that increases exponentially per digit below the spam threshold, say 20% to send 0.001BTC, 50% for 0.0001BTC, 100% for 0.00001BTC, and doubling forward. Of course, since this was very poorly thought out before client was released, figuring out how to reach any kind of consensus between miners and clients is not going to be easy.

I won't spam up this thread anymore, but I think what you are proposing has serious flaws. Since the blockchain is a "tragedy of the commons" type of problem, unless all miners and all clients agree to what a reasonable fee is, there's going to be serious conflicts at some point. Either significant numbers of empty blocks, no way for a client to determine what a "good" fee is, and if enough people get tired of that behavior, you might end up on your very own blockchain fork one day. Of course, for the protocol to change at all, and for any reason would generally end up producing large forks, which is a major downside of the BTC system, and of the fact that it's experimental.

I'm So Meta, Even This Acronym
rjk
Sr. Member
****
Offline Offline

Activity: 448
Merit: 250


1ngldh


View Profile
March 26, 2012, 07:26:27 PM
 #26

".01 BTC" is not a reasonable fee, period. Here's why.

I lend 1BTC to someone for 1 day, interest paid is 1%, total 1.01 BTC due.

I pay him the 1BTC loan, there's 0.01BTC paid to you.

He spends his 1BTC on whatever it is he wanted, there's 0.01 BTC paid to you.

He trades cash for 1.01BTC to pay me back, another 0.01 BTC paid to you.

He sends me the 1.01BTC, there's another .01BTC paid to you.

The interest on the loan was 0.01 BTC, total interest paid to me = 0.01BTC, interest paid to you just for the tx, .04 BTC, or 4 times the interest the loan was worth. This sort of 1BTC/1% transaction happens regularly on the lending forum.

I should point out that western union does indeed charge based on how much you send, although it's a flat rate for up to a certain amount, but that's pretty irrelevant. I would say a "Fair" rate is more like 0.05%, maybe 0.025%, with a spam fee that increases exponentially per digit below the spam threshold, say 20% to send 0.001BTC, 50% for 0.0001BTC, 100% for 0.00001BTC, and doubling forward. Of course, since this was very poorly thought out before client was released, figuring out how to reach any kind of consensus between miners and clients is not going to be easy.

I won't spam up this thread anymore, but I think what you are proposing has serious flaws. Since the blockchain is a "tragedy of the commons" type of problem, unless all miners and all clients agree to what a reasonable fee is, there's going to be serious conflicts at some point. Either significant numbers of empty blocks, no way for a client to determine what a "good" fee is, and if enough people get tired of that behavior, you might end up on your very own blockchain fork one day. Of course, for the protocol to change at all, and for any reason would generally end up producing large forks, which is a major downside of the BTC system, and of the fact that it's experimental.
If you are dealing with piddly small amounts like that, you can deal with waiting longer for fee-free transactions to be processed by generous nodes.

Mining Rig Extraordinaire - the Trenton BPX6806 18-slot PCIe backplane [PICS] Dead project is dead, all hail the coming of the mighty ASIC!
Luke-Jr
Legendary
*
Offline Offline

Activity: 2576
Merit: 1186



View Profile
March 26, 2012, 07:35:10 PM
 #27

a) provide p2pool users some control
Hey, don't forget BitPenny and Eligius miners! Wink

Now is there a guide anywhere for compiling modified version of bitcoind under windows?
If you want, I could trivially build a Windows binary. I won't bill for the time spent by the compiler, of course :p

".01 BTC" is not a reasonable fee, period. Here's why.
Your argument seems to be based entirely on trying to profit from usury. Usury is evil, so IMO your argument can be ignored. Wink

Matt Corallo
Hero Member
*****
Offline Offline

Activity: 755
Merit: 515


View Profile
March 26, 2012, 07:36:56 PM
 #28

".01 BTC" is not a reasonable fee, period. Here's why.
That is far from the point.  The point isnt to decide what is and isnt a reasonable fee, the point is to see what the market will shift towards as subsidy decreases.  And I have yet to hear a reason why it wouldnt shift down quite low at that time.

Bitcoin Core, rust-lightning, http://bitcoinfibre.org etc.
PGP ID: 07DF 3E57 A548 CCFB 7530  7091 89BB B866 3E2E65CE
rjk
Sr. Member
****
Offline Offline

Activity: 448
Merit: 250


1ngldh


View Profile
March 26, 2012, 07:43:41 PM
 #29

I won't bill for the time spent by the compiler, of course :p

Mining Rig Extraordinaire - the Trenton BPX6806 18-slot PCIe backplane [PICS] Dead project is dead, all hail the coming of the mighty ASIC!
Haplo
Full Member
***
Offline Offline

Activity: 168
Merit: 100



View Profile
March 26, 2012, 07:47:04 PM
 #30

Your argument seems to be based entirely on trying to profit from usury. Usury is evil, so IMO your argument can be ignored. Wink

Lol, well then my resources stay in fiat or PMs. Tongue

That is far from the point.  The point isnt to decide what is and isnt a reasonable fee, the point is to see what the market will shift towards as subsidy decreases.  And I have yet to hear a reason why it wouldn't shift down quite low at that time.

How are market values determined? Except by what the miners who control the majority processing power arbitrarily decide? In any other economic situation the trade between price and willing customers would necessarily lead to fair market prices, but here if a miner creates a block, then they basically get to decide the network's fee policy for the next 10 minutes all to themselves. That might be somewhat reasonable at a 0% subsidy, but at current that could easily kill the network if a majority of miners happen to be stupid (which is guaranteed in the long run). With no way for a fair market price to be determined by supply and demand, miners become parasites who decide on their own welfare checks, and users become orphaned trying to discover what fee will reasonably get their tx pushed.

I'm So Meta, Even This Acronym
DeathAndTaxes (OP)
Donator
Legendary
*
Offline Offline

Activity: 1218
Merit: 1063


Gerald Davis


View Profile
March 26, 2012, 07:48:59 PM
 #31

".01 BTC" is not a reasonable fee, period. Here's why.

That is how free markets work.  If I want to change 0.01 BTC I can.  Obviously I lose your business.  Someone else might charge 0.005, someone else 0.001, someone else might accept free tx.

There is no scenario where every miner demands the exact same fee amount.

Hypothetically someday lets say:

10% of miners require a fee of at least 0.01 BTC
25% of miners require a fee of at least 0.005 BTC
25% of miners require a fee of at least 0.0001 BTC
20% of miners require a fee of at least a single satoshi.
20% of miners require no fee.

Obviously in reality this sliding scale would have many more data points but using this as an example.

If you included no fee your tx would still be processed but only 20% of the network would be working on it so confirmation time is 50 min.
If you included a single satoshi as a fee then 40% of the network would be working on it so your confirmation time drops to only 25 min.
If you included a fee of 0.0001 (1/20th of a cent at todays value) you buy 65% of the network and avg confirmation of 15 min.
etc.

If miners broadcasted fee schedules and hashing power via a p2p protocol clients could advise users on fees and projected confirmation times.

Another option would be to include a "charity window".  Miner could specify to include x  or x% of outstanding tx which are below the fee limit.  "low fee" tx would be processed first come first serve after priority tx so confirmation times would be longer but they would be processed.

Quote
I would say a "Fair" rate is more like 0.05%, maybe 0.025%, with a spam fee that increases exponentially per digit below the spam threshold, say 20% to send 0.001BTC, 50% for 0.0001BTC, 100% for 0.00001BTC, and doubling forward. Of course, since this was very poorly thought out before client was released, figuring out how to reach any kind of consensus between miners and clients is not going to be easy.

% based fees are neither desirable nor possible in the Bitcoin network.  What you or I consider fair is irrelevant.  Most miners will likely accept 0 BTC tx, each if free to set their own price.  Users are free to set what they are willing to pay.  The market (not you or me) will determine what is "fair".

Fair price in a free market is what someone is willing to pay and someone is willing to sell.  What is being bought and sold is inclusion of a tx in the next block.


Quote
unless all miners and all clients agree to what a reasonable fee is, there's going to be serious conflicts at some point. Either significant numbers of empty blocks, no way for a client to determine what a "good" fee is
That is simply impossible.  100% consensus on "fair" never happens.  Only central control can implement price controls.  IF you feel that is the only possible "fix" then you should uninstall Bitcoin now because it will NEVER happen in a decentralized network.


Quote
and if enough people get tired of that behavior, you might end up on your very own blockchain fork one day. Of course, for the protocol to change at all, and for any reason would generally end up producing large forks, which is a major downside of the BTC system, and of the fact that it's experimental.

This requires no protocol change.  If hypothetically miners setting their own tx fees was outlawed by some future protocol change well I could simply stop (and I likely would stop mining too).
Polvos
Hero Member
*****
Offline Offline

Activity: 597
Merit: 500



View Profile
March 26, 2012, 08:31:39 PM
 #32

Polvos, you also offered to contribute toward the bounty; if you feel the desire to tip me, you can use 1DGzpZzce1c7nsg1SN7exV6bsaju1Mcrc6 - it's fine with me if you choose not to, though.

Let's see how works your code and I will tip you some coins. For the moment it seems quite promising. Good work.

Haplo
Full Member
***
Offline Offline

Activity: 168
Merit: 100



View Profile
March 26, 2012, 08:55:09 PM
 #33

Nevermind, I can see how % rates are unworkable now.

I think pay-per-KB or per byte or whatever is reasonable, but I think it needs a little more tweaking so the whole network can be informed about what fees miners are charging. I think that a per-KB fee should also make spam fees obsolete.

The only problem that I can see is that, as per the scenario you outlined, fees might vary by several orders of magnitude, which might make it a huge pain for a client to decide on a fee. Is it possible for a client to determine what % of hashing power is charging what? If you can figure that out, then you could create a "confidence scale" to match a minimum fee with a minimum confidence rate.

I'm So Meta, Even This Acronym
DeathAndTaxes (OP)
Donator
Legendary
*
Offline Offline

Activity: 1218
Merit: 1063


Gerald Davis


View Profile
March 26, 2012, 08:56:33 PM
 #34

If you want, I could trivially build a Windows binary. I won't bill for the time spent by the compiler, of course :p

That would be great.  Thanks.
Luke-Jr
Legendary
*
Offline Offline

Activity: 2576
Merit: 1186



View Profile
March 26, 2012, 10:40:25 PM
 #35

Windows binaries are done: installer | zip (sigs)

DeathAndTaxes (OP)
Donator
Legendary
*
Offline Offline

Activity: 1218
Merit: 1063


Gerald Davis


View Profile
March 27, 2012, 02:27:04 PM
 #36

Thanks for the binaries.

I had a problem when copying existing blockchain & wallet (which is empty) from RC2 version.  Yeah stupidly I didn't write down the error.  I cleared the data directory and bitcoind worked fine from a coldstart (created new wallet and downloaded entire blockcahin).

The modified bitcoind has been powering my p2pool instance for 10 hours now and p2pool hasn't reported any errors and I see no change in sharechain deads, rejects, or orphans. 

I adjusted fee to 0.01 BTC, to zero, and to 1 satoshi.  For all values getmininginfo showed set minfee properly and getmemorypool returned the correct transactions (decoding them to verify was "fun" Smiley ).  

For anyone interested I will post some screenshots this evening and provide an update.  Given I only have 15GH/s it likely will take a while before I see a block from the custom memorypool but p2pool doesn't have a problem with the memorypool returned.

Thanks for your help Luke.  I have some ideas to build upon this in the future that may alleviate some of the (unecessary IMHO) concern.

One related question.  Is it possible to have bitcoind create a tx but not broadcast it?  If so is it then part of the memorypool?  If not could bitcoind be modified to make it part of the memorypool?  Essentially I want the ability to include a tx in a block that hasn't been broadcasted (coin melting).
Luke-Jr
Legendary
*
Offline Offline

Activity: 2576
Merit: 1186



View Profile
March 27, 2012, 02:35:02 PM
 #37

One related question.  Is it possible to have bitcoind create a tx but not broadcast it?  If so is it then part of the memorypool?  If not could bitcoind be modified to make it part of the memorypool?  Essentially I want the ability to include a tx in a block that hasn't been broadcasted (coin melting).
Possible, but might not be simple to do in bitcoind.

marked
Full Member
***
Offline Offline

Activity: 168
Merit: 100



View Profile
March 27, 2012, 04:26:39 PM
 #38

I'd like to make a suggestion that a mining pool announces the TX fee to the network.

The announcement is a simple message containing a pool id, a fee, and a txfee duration. The message is signed with the pool public key.

The fee is valid for the next "n" blocks where "n" is a value determined to be no less than 6 valid tx blocks, and no more than 6*24*30 blocks.

  • 6 valid TX block to bypass the 0TX miner otherwise it is theoretically possible to send a tx and have a new fee structure, so the tx gets dropped.
  • Client's would need to be modified to listen for this message. A set of known public keys to decrypt the message or authenticate it would be required from each mining pool and included in each client.
  • If a block is mined with a non-announcing pool/miner then the fee is set to a default.

So for example
deathandtaxes mines with a min fee of 0.01
slush mines with a min fee of 0.005
eclipsemc mines with a min fee of 0.001
deepbit mines with a min fee of 0.0005
eligius mines with a min fee of 0.0002

(not casting doubt on hashing power, but just choosing pools from memory and in no particular order.)

In this instance my client can choose

to be somewhat lax, and have it on a when it gets confirmed if at all basis with a fee of <0.0002,
to get the somewhere during this day, at 0.0002
to get confirmed within a half day, at 0.0005
to get confirmed within 3 hours, at 0.001
to get confirmed within 1 hour at 0.005
to get confirmed in the next valid tx block at 0.01 (because all mining block finders would include under that structure)

thoughts?

marked
DeathAndTaxes (OP)
Donator
Legendary
*
Offline Offline

Activity: 1218
Merit: 1063


Gerald Davis


View Profile
March 27, 2012, 04:32:40 PM
 #39

I do think something like that will be needed in time so the client can make informed decisions on confirmation time vs cost.

It might be a little early adding that to the protocol but given fees can be dynamic that information will need to be shared and the network already has the infrastructure for relaying information in a p2p fashion.  The one thing I am not sure about is that for the info to be useful you would want proof of hashing power.  It doesn't matter what % of nodes say they require a fee of 0.0025 what matters is what % of the hashing power does. 

My first thought is that the protocol could be modified so that the fee data is added to block (coinbase?).  That way the client can look at the last x blocks and get an estimate of  the fees the network is requiring.  That combined with the number of pending tx and their included fees would allow the client to make educated guesses on confirmation times.

Still we likely are years away from needing such a protocol enhancement but it is good to start thinking about it.
Luke-Jr
Legendary
*
Offline Offline

Activity: 2576
Merit: 1186



View Profile
March 27, 2012, 04:44:02 PM
 #40

My first thought is that the protocol could be modified so that the fee data is added to block (coinbase?).  That way the client can look at the last x blocks and get an estimate of  the fees the network is requiring.  That combined with the number of pending tx and their included fees would allow the client to make educated guesses on confirmation times.
That wouldn't allow miners to change fees quickly. IMO, a better solution is to push a URI and signing pubkeyhash into the coinbase... then have a simple (signed) script at the URI that calculates the fee required for a given txn.

Pages: « 1 [2] 3 »  All
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!